• Welcome to Oodle
  • About Oodle
    • Index of Abouts
    • About Oodle ozip
    • About Oodle on Platforms
      • About Oodle on Windows
        • About Oodle on Windows UWP
      • About Oodle on PS4
      • About Oodle on Nintendo Switch
      • About Oodle on Mac
      • About Oodle on Xbox One
      • About Oodle on Linux
      • About Oodle on IOS
      • About Oodle on Android
      • About Oodle on WASM
    • About Oodle Job Threading Plugins
    • About Compression Scratch Memory
  • Frequently Asked Questions
    • FAQ: OodleLZ_Decompress is failing how do I diagnose it?
    • FAQ: How do I use Oodle with no allocator?
    • FAQ: How do I limit the encoder memory use?
    • FAQ: How much memory do the Oodle compressors use ?
    • FAQ: What are the Oodle deprecated compressors ?
    • FAQ: Do new Oodle versions break data compatibility ?
    • FAQ: How does OodleLZ compare with other compressors ?
    • FAQ: Which OodleLZ should I use?
    • FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
    • FAQ: How do I decompress to graphics memory quickly?
    • FAQ: How do I get the Oodle logs?
    • FAQ: I write a file with IOQ but the contents are garbage?
    • FAQ: I ran out of OodleHandle table slots; what do I do?
    • FAQ: What is SINTa? How do I load files bigger than 2 GB?
    • FAQ: My Files aren't loading right and I can't track it down
  • Oodle2 Core API Documentation
    • Core Base
      • Defines
        • OODLE_MALLOC_MINIMUM_ALIGNMENT
        • OODLE_JOB_MAX_DEPENDENCIES
        • OODLE_JOB_NULL_HANDLE
        • t_fp_Oodle_Job
        • OODLE_HEADER_VERSION
        • OodleNetworkVersion
      • Enumerants
        • Oodle_UsageWarnings
      • Structures
        • OodleConfigValues
      • Functions
        • Oodle_GetConfigValues
        • Oodle_SetConfigValues
        • Oodle_SetUsageWarnings
        • Oodle_CheckVersion
        • Oodle_LogHeader
      • Typedefs
        • t_OodleFPVoidVoid
        • t_OodleFPVoidVoidStar
    • Core plugins
      • Functions
        • OodleCore_Plugins_SetAllocators
        • OodleCore_Plugins_SetJobSystem
        • OodleCore_Plugins_SetJobSystemAndCount
        • OodleCore_Plugins_SetPrintf
        • OodleCore_Plugins_SetAssertion
      • Typedefs
        • t_fp_OodleCore_Plugin_MallocAligned
        • t_fp_OodleCore_Plugin_Free
        • t_fp_OodleCore_Plugin_RunJob
        • t_fp_OodleCore_Plugin_WaitJob
        • t_fp_OodleCore_Plugin_Printf
        • t_fp_OodleCore_Plugin_DisplayAssertion
    • Core LZ compression
      • About OodleLZ
      • About OodleLZ ThreadPhased Decode
      • About OodleLZ Hydra
      • OodleAPI_LZ_Compressors
        • Defines
          • OODLE_ALLOW_DEPRECATED_COMPRESSORS
          • OODLELZ_LOCALDICTIONARYSIZE_MAX
          • OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
          • OODLELZ_BLOCK_LEN
          • OODLELZ_BLOCK_MAX_COMPLEN
          • OODLELZ_QUANTUM_LEN
          • OODLELZ_FAILED
          • OODLELZ_SCRATCH_MEM_NO_BOUND
        • Enumerants
          • OodleLZ_Verbosity
          • OodleLZ_Compressor
          • OodleLZ_PackedRawOverlap
          • OodleLZ_CheckCRC
          • OodleLZ_Profile
          • OodleDecompressCallbackRet
          • OodleLZ_CompressionLevel
          • OodleLZ_Jobify
          • OodleLZ_Decode_ThreadPhase
          • OodleLZ_FuzzSafe
          • OodleLZSeekTable_Flags
          • OodleLZ_CompressScratchMemBoundType
        • Structures
          • OodleLZ_CompressOptions
          • OodleLZ_DecodeSome_Out
          • OodleLZDecoder
          • OodleLZ_SeekTable
        • Functions
          • OodleLZ_Compress
          • OodleLZ_Decompress
          • OodleLZDecoder_Create
          • OodleLZDecoder_MemorySizeNeeded
          • OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
          • OodleLZDecoder_Destroy
          • OodleLZDecoder_Reset
          • OodleLZDecoder_DecodeSome
          • OodleLZDecoder_MakeValidCircularWindowSize
          • OodleLZ_MakeSeekChunkLen
          • OodleLZ_GetNumSeekChunks
          • OodleLZ_GetSeekTableMemorySizeNeeded
          • OodleLZ_FillSeekTable
          • OodleLZ_CreateSeekTable
          • OodleLZ_FreeSeekTable
          • OodleLZ_CheckSeekTableCRCs
          • OodleLZ_FindSeekEntry
          • OodleLZ_GetSeekEntryPackedPos
          • OodleLZ_CompressionLevel_GetName
          • OodleLZ_Compressor_GetName
          • OodleLZ_Jobify_GetName
          • OodleLZ_CompressOptions_GetDefault
          • OodleLZ_CompressOptions_Validate
          • OodleLZ_Compressor_UsesWholeBlockQuantum
          • OodleLZ_Compressor_UsesLargeWindow
          • OodleLZ_Compressor_CanDecodeInCircularWindow
          • OodleLZ_Compressor_CanDecodeThreadPhased
          • OodleLZ_Compressor_CanDecodeInPlace
          • OodleLZ_Compressor_MustDecodeWithoutResets
          • OodleLZ_Compressor_CanDecodeFuzzSafe
          • OodleLZ_Compressor_RespectsDictionarySize
          • OodleLZ_GetCompressScratchMemBound
          • OodleLZ_GetCompressScratchMemBoundEx
          • OodleLZ_GetCompressedBufferSizeNeeded
          • OodleLZ_GetDecodeBufferSize
          • OodleLZ_GetInPlaceDecodeBufferSize
          • OodleLZ_GetCompressedStepForRawStep
          • OodleLZ_GetAllChunksCompressor
          • OodleLZ_GetFirstChunkCompressor
          • OodleLZ_GetChunkCompressor
        • Typedefs
          • OodleDecompressCallback
  • Oodle2 Ext API Documentation
    • OodleX LZ compression
      • OodleXAPI_LZ_Async
        • Structures
          • OodleDecompressCallback_WriteFile_Data
        • Functions
          • OodleXLZ_Decompress_ThreadPhased_Narrow_Async
          • OodleXLZ_Decompress_Narrow_Async
          • OodleXLZ_ReadAndDecompress_Wide_Async
          • OodleXLZ_Decompress_Wide_Async
          • OodleXLZ_Decompress_MakeSeekTable_Wide_Async
          • OodleXLZ_Compress_Async
          • OodleXLZ_Compress_Wait_GetResult
          • OodleXLZ_Compress_AsyncAndWait
          • OodleXLZ_ReadAndDecompress_Stream_Async
          • OodleXDecompressCallback_WriteFile_Data_Init
          • OodleDecompressCallback_WriteFile
    • OodleX Startup and Shutdown
      • Defines
        • OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
        • OODLE_WORKERS_COUNT_ALL_HYPER_CORES
      • Enumerants
        • OodleX_Init_GetDefaults_DebugSystems
        • OodleX_Init_GetDefaults_Threads
        • OodleX_Shutdown_LogLeaks
        • OodleX_Shutdown_DebugBreakOnLeaks
      • Structures
        • OodleXInitOptions
        • OodleXConfigValues
      • Functions
        • OodleX_Init_ThreadProfilerInit
        • OodleX_Init_GetDefaults
        • OodleX_Init_GetDefaults_Minimal
        • OodleX_Init
        • OodleX_Init_Default
        • OodleX_LogSystemInfo
        • OodleX_Shutdown
        • OodleX_Init_NoThreads
        • OodleX_Shutdown_NoThreads
        • OodleX_GetConfigValues
        • OodleX_SetConfigValues
    • OodleX Memory Allocators
      • About OodleXMalloc
      • Enumerants
        • OodleXMalloc_OS_Options
      • Structures
        • OodleXMallocVTable
      • Functions
        • OodleXMalloc_InstallVTable
        • OodleXMalloc_SetFailedHandler
        • OodleXMalloc
        • OodleXMallocAligned
        • OodleXFree
        • OodleXFreeSized
        • OodleXMallocBigAlignment
        • OodleXMallocBig
        • OodleXFreeBig
        • OodleXMalloc_ValidatePointer
        • OodleXMalloc_IOAligned
        • OodleXFree_IOAligned
        • OodleXMalloc_GetVTable_Clib
        • OodleXMalloc_GetVTable_OS
      • Typedefs
        • OodleXMallocFailedHandler
    • OodleX async handle operations
      • About OodleXHandle
      • Defines
        • OODLEX_ASYNC_HANDLE_INVALID
        • OODLEX_ASYNC_HANDLE_PENDING
        • OODLEX_ASYNC_HANDLE_DONE
        • OODLEX_ASYNC_HANDLE_ERROR
      • Enumerants
        • OodleXPriority
        • OodleXAsyncSelect
        • OodleXStatus
        • OodleXHandleAutoDelete
        • OodleXHandleKickDelayed
        • OodleXHandleDeleteIfDone
      • Functions
        • OodleX_GetStatus
        • OodleX_Wait
        • OodleX_WaitAll
        • OodleX_WaitDoneAllPending
        • OodleX_SetHandleAutoDelete
        • OodleX_GetAvailableAsyncSelect
        • OodleXHandleEvent_Alloc
        • OodleXHandleEvent_SetDone
        • OodleXHandleEvent_SetError
        • OodleXHandleCountdown_Alloc
        • OodleXHandleCountdown_Decrement
      • Typedefs
        • OodleXHandle
    • OodleX threading util
      • Functions
        • OodleX_Semaphore_Post
        • OodleX_Semaphore_Wait
        • OodleX_CreateThread
        • OodleX_WaitAndDestroyThread
        • OodleX_ReleaseThreadTLS
        • OodleX_CorePlugin_RunJob
        • OodleX_CorePlugin_WaitJob
        • OodleX_GetNumWorkerThreads
      • Typedefs
        • OodleX_Semaphore
        • OodleX_ThreadFunc
    • OodleX low level async io
      • About OodleIOQ
      • Defines
        • OODLEX_IO_MAX_ALIGNMENT
        • OODLEX_BUFFER_SIZE_DEFAULT
        • OODLEX_FILEINFO_FLAG_INVALID
        • OODLEX_FILEINFO_MODTIME_INVALID
        • OODLEX_FILE_SIZE_INVALID
        • OODLEX_FILE_OPEN_NO_RESERVE_SIZE
        • OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
      • Enumerants
        • OodleXCopyFileFlags
        • OodleFileNotFoundIsAnError
        • OODLEX_FILEINFO_FLAGS
        • OodleXError
        • OodleXFileMode
        • OodleXFileOpenFlags
      • Structures
        • OodleXFileOpsVTable
        • OodleXFileInfo
      • Functions
        • OodleXIOQ_WaitDoneAllPending
        • OodleXIOQ_KickAnyDelayed
        • OodleXIOQ_GetStatus
        • OodleXIOQ_GetErrorDetails
        • OodleXIOQ_GetErrorEnum
        • OodleXIOQ_LogError
        • OodleXIOQ_Wait
        • OodleXIOQ_GetInfo
        • OodleXIOQ_Wait_GetInfo
        • OodleXIOQ_GetLastPendingOpOnFile
        • OodleXIOQ_GetName
        • OodleXIOQ_GetLastError
        • OodleXIOQ_ClearError
        • OodleXIOQ_LogLastError
        • OodleXIOQ_GetOSHandle
        • OodleXIOQ_SetVTable
        • OodleXIOQ_Fence_Async
        • OodleXIOQ_OpenForRead_Async
        • OodleXIOQ_OpenAndRead_Async
        • OodleXIOQ_OpenForWriteCreate_Async
        • OodleXIOQ_OpenForWriteTempName_Async
        • OodleXIOQ_CloseFile_Async
        • OodleXIOQ_CloseFileRename_Async
        • OodleXIOQ_Read_Async
        • OodleXIOQ_Write_Async
        • OodleXIOQ_SetFileSize_Async
        • OodleXIOQ_ReserveFileSizeForWrite_Async
        • OodleXIOQ_ForceWriteable_Async
        • OodleXIOQ_Delete_Async
        • OodleXIOQ_Rename_Async
        • OodleXIOQ_MakeDir_Async
        • OodleXIOQ_FreeBufferIOAligned_Async
        • OodleXIOQ_GetInfoByName_Async
        • OodleXIOQ_GetInfoByName_GetResult
        • OodleXIOQ_SetInfoByName_Async
        • OodleXIOQ_ReadMallocWholeFile_Async
        • OodleXIOQ_ReadMallocWholeFile_GetResult
        • OodleXIOQ_OpenAndReadMallocWholeFile_Async
        • OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
        • OodleXIOQ_OpenWriteWholeFileClose_Async
        • OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
        • OodleXIOQ_ReadUnalignedAdjustPointer_Async
        • OodleXIOQ_MakeAllDirs_Async
        • OodleXIOQ_CopyFile_Async
        • OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
        • OodleXIOQ_WriteWholeFile_AsyncAndWait
        • OodleXIOQ_CopyFile_AsyncAndWait
        • OodleXIOQ_GetInfoByName_AsyncAndWait
        • OodleXIOQ_SetInfoByName_AsyncAndWait
        • OodleXIOQ_MakeAllDirs_AsyncAndWait
        • OodleXIOQ_Delete_AsyncAndWait
        • OodleXIOQ_Rename_AsyncAndWait
        • OodleXIOQ_GetFileSize_AsyncAndWait
        • OodleXIOQ_NameIsDir_AsyncAndWait
        • OodleX_GetOSFileOps
        • OodleX_GetDefaultFileOps
        • OodleX_SetDefaultFileOps
      • Typedefs
        • OodleXIOQFile
        • OodleXOSFile
        • OodleXOSFileListing
    • OodleX Debug aids
      • Defines
        • OodleXLog_Printf
      • Enumerants
        • OodleXLog_StateFlags
        • OodleXLog_VerboseLevel
        • OodleXLogCallbackRetRet
      • Functions
        • OodleXLog_SetState
        • OodleXLog_SetEcho
        • OodleXLog_GetEcho
        • OodleXLog_SetCallback
        • OodleXLog_GetCallback
        • OodleXLog_GetVerboseLevel
        • OodleXLog_SetVerboseLevel
        • OodleXLog_Flush
        • OodleXLog_PrintfError
      • Typedefs
        • OodleXLogCallbackRet
    • OodleX Utils
      • Defines
        • OODLEX_PATH_DELIM
      • Functions
        • OodleX_GetExtensionKey
        • OodleX_MakeExtensionKey
        • OodleX_IOAlignUpS32
        • OodleX_IOAlignUpS64
        • OodleX_IOAlignUpSINTa
        • OodleX_IOAlignDownS32
        • OodleX_IOAlignDownS64
        • OodleX_IOAlignDownSINTa
        • OodleX_S64_to_SINTa_check
        • OodleX_GetSeconds
        • OodleXUtil_ConvertUTF8ToUTF16
        • OodleXUtil_ConvertUTF16ToUTF8
        • OodleX_CombinePaths
        • OodleX_GetOSCwd
        • OodleX_PrefixOSCwd
    • About Oodle Ext
  • Oodle2 Network API Documentation
    • Oodle Network compression
      • OodleAPI_OodleNetwork1
        • About OodleNetwork1
        • Defines
          • OODLENETWORK1_MAX_DICTIONARY_SIZE
          • OODLENETWORK1_HASH_BITS_DEFAULT
          • OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
        • Structures
          • OodleNetwork1_Shared
          • OodleNetwork1TCP_State
          • OodleNetwork1UDP_State
          • OodleNetwork1UDP_StateCompacted
        • Functions
          • OodleNetwork1_Shared_Size
          • OodleNetwork1TCP_State_Size
          • OodleNetwork1_CompressedBufferSizeNeeded
          • OodleNetwork1_Shared_SetWindow
          • OodleNetwork1TCP_State_Reset
          • OodleNetwork1TCP_State_InitAsCopy
          • OodleNetwork1TCP_Train
          • OodleNetwork1TCP_Encode
          • OodleNetwork1TCP_Decode
          • OodleNetwork1UDP_Train
          • OodleNetwork1UDP_State_Size
          • OodleNetwork1UDP_Encode
          • OodleNetwork1UDP_Decode
          • OodleNetwork1UDP_StateCompacted_MaxSize
          • OodleNetwork1UDP_State_Compact_ForVersion
          • OodleNetwork1UDP_State_Compact
          • OodleNetwork1UDP_State_Uncompact_ForVersion
          • OodleNetwork1UDP_State_Uncompact
          • OodleNetwork1_SelectDictionarySupported
          • OodleNetwork1_SelectDictionaryFromPackets
          • OodleNetwork1_SelectDictionaryFromPackets_Trials
      • About Oodle Network Compression
        • Capturing Training data for OodleNetwork
        • Forming Packets for Maximum Compression
    • Network plugins
      • Functions
        • OodleNet_Plugins_SetAllocators
        • OodleNet_Plugins_SetJobSystem
        • OodleNet_Plugins_SetJobSystemAndCount
        • OodleNet_Plugins_SetPrintf
        • OodleNet_Plugins_SetAssertion
      • Typedefs
        • t_fp_OodleNet_Plugin_MallocAligned
        • t_fp_OodleNet_Plugin_Free
        • t_fp_OodleNet_Plugin_RunJob
        • t_fp_OodleNet_Plugin_WaitJob
        • t_fp_OodleNet_Plugin_Printf
        • t_fp_OodleNet_Plugin_DisplayAssertion
  • Examples
    • How to build and use the Oodle examples
    • example_lz : Example demonstrating LZ compression and decompression
    • example_lz_chart : Example that makes a chart of OodleLZ options
    • example_lz_noallocs : Example demonstrating Oodle compression with no allocations
    • example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
    • example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
    • example_lz_threadphased : Example of 2-thread ThreadPhased decoding
    • example_network_client : Example with simple network client support
    • example_packet : Example demonstrating network packet compression
  • Acknowledgements
    • Third Party License Notices
  • Change Log
  • Getting Started with Oodle Network
  • Introducing the new Oodle Leviathan
  • Oodle2 Core vs Oodle2 Ext
  • Tips for benchmarking a compressor
  • Getting Started with Oodle LZ Data Compression
  • Oodle Network and Data SDK separation
  • About Compression Scratch Memory
  • About Oodle
  • About Oodle Ext
  • About Oodle Job Threading Plugins
  • About Oodle Network Compression
  • About Oodle on Android
  • About Oodle on IOS
  • About Oodle on Linux
  • About Oodle on Mac
  • About Oodle on Nintendo Switch
  • About Oodle on PS4
  • About Oodle on Platforms
  • About Oodle on WASM
  • About Oodle on Windows
  • About Oodle on Windows UWP
  • About Oodle on Xbox One
  • About Oodle ozip
  • About OodleIOQ
  • About OodleLZ
  • About OodleLZ Hydra
  • About OodleLZ ThreadPhased Decode
  • About OodleNetwork1
  • About OodleXHandle
  • About OodleXMalloc
  • Acknowledgements
  • Capturing Training data for OodleNetwork
  • Change Log
  • Core Base
  • Core LZ compression
  • Core plugins
  • Examples
  • FAQ: Do new Oodle versions break data compatibility ?
  • FAQ: How do I decompress to graphics memory quickly?
  • FAQ: How do I get the Oodle logs?
  • FAQ: How do I limit the encoder memory use?
  • FAQ: How do I use Oodle with no allocator?
  • FAQ: How does OodleLZ compare with other compressors ?
  • FAQ: How much memory do the Oodle compressors use ?
  • FAQ: I ran out of OodleHandle table slots; what do I do?
  • FAQ: I write a file with IOQ but the contents are garbage?
  • FAQ: My Files aren't loading right and I can't track it down
  • FAQ: OodleLZ_Decompress is failing how do I diagnose it?
  • FAQ: What are the Oodle deprecated compressors ?
  • FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
  • FAQ: What is SINTa? How do I load files bigger than 2 GB?
  • FAQ: Which OodleLZ should I use?
  • Forming Packets for Maximum Compression
  • Frequently Asked Questions
  • Getting Started with Oodle LZ Data Compression
  • Getting Started with Oodle Network
  • How to build and use the Oodle examples
  • Index of Abouts
  • Introducing the new Oodle Leviathan
  • Network plugins
  • OODLELZ_BLOCK_LEN
  • OODLELZ_BLOCK_MAX_COMPLEN
  • OODLELZ_FAILED
  • OODLELZ_LOCALDICTIONARYSIZE_MAX
  • OODLELZ_QUANTUM_LEN
  • OODLELZ_SCRATCH_MEM_NO_BOUND
  • OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
  • OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
  • OODLENETWORK1_HASH_BITS_DEFAULT
  • OODLENETWORK1_MAX_DICTIONARY_SIZE
  • OODLEXLOG_AUTOFLUSH_THREADLOG
  • OODLEXLOG_CALLBACK
  • OODLEXLOG_ECHO
  • OODLEXLOG_FILE_LINE
  • OODLEXLOG_FLUSH_EVERY_WRITE
  • OODLEXLOG_PREFIX_THREAD_TIME
  • OODLEXLOG_STATE_VERBOSITY0
  • OODLEXLOG_STATE_VERBOSITY1
  • OODLEXLOG_STATE_VERBOSITY2
  • OODLEXLOG_STATE_VERBOSITY_NONE
  • OODLEXLOG_TO_DEBUGGER
  • OODLEXLOG_TO_FILE
  • OODLEX_ASYNC_HANDLE_DONE
  • OODLEX_ASYNC_HANDLE_ERROR
  • OODLEX_ASYNC_HANDLE_INVALID
  • OODLEX_ASYNC_HANDLE_PENDING
  • OODLEX_BUFFER_SIZE_DEFAULT
  • OODLEX_FILEINFO_FLAGS
  • OODLEX_FILEINFO_FLAG_DIR
  • OODLEX_FILEINFO_FLAG_Force32
  • OODLEX_FILEINFO_FLAG_HIDDEN
  • OODLEX_FILEINFO_FLAG_INVALID
  • OODLEX_FILEINFO_FLAG_OFFLINE
  • OODLEX_FILEINFO_FLAG_READONLY
  • OODLEX_FILEINFO_FLAG_SYMLINK
  • OODLEX_FILEINFO_FLAG_TEMPORARY
  • OODLEX_FILEINFO_MODTIME_INVALID
  • OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
  • OODLEX_FILE_OPEN_NO_RESERVE_SIZE
  • OODLEX_FILE_SIZE_INVALID
  • OODLEX_IO_MAX_ALIGNMENT
  • OODLEX_PATH_DELIM
  • OODLE_ALLOW_DEPRECATED_COMPRESSORS
  • OODLE_HEADER_VERSION
  • OODLE_JOB_MAX_DEPENDENCIES
  • OODLE_JOB_NULL_HANDLE
  • OODLE_MALLOC_MINIMUM_ALIGNMENT
  • OODLE_WORKERS_COUNT_ALL_HYPER_CORES
  • OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
  • Oodle Network and Data SDK separation
  • Oodle Network compression
  • Oodle2 Core API Documentation
  • Oodle2 Core vs Oodle2 Ext
  • Oodle2 Ext API Documentation
  • Oodle2 Network API Documentation
  • OodleAPI_LZ_Compressors
  • OodleAPI_OodleNetwork1
  • OodleConfigValues
  • OodleCore_Plugins_SetAllocators
  • OodleCore_Plugins_SetAssertion
  • OodleCore_Plugins_SetJobSystem
  • OodleCore_Plugins_SetJobSystemAndCount
  • OodleCore_Plugins_SetPrintf
  • OodleDecompressCallback
  • OodleDecompressCallbackRet
  • OodleDecompressCallbackRet_Cancel
  • OodleDecompressCallbackRet_Continue
  • OodleDecompressCallbackRet_Force32
  • OodleDecompressCallbackRet_Invalid
  • OodleDecompressCallback_WriteFile
  • OodleDecompressCallback_WriteFile_Data
  • OodleFileNotFoundIsAnError
  • OodleFileNotFoundIsAnError_Force32
  • OodleFileNotFoundIsAnError_No
  • OodleFileNotFoundIsAnError_Yes
  • OodleLZDecoder
  • OodleLZDecoder_Create
  • OodleLZDecoder_DecodeSome
  • OodleLZDecoder_Destroy
  • OodleLZDecoder_MakeValidCircularWindowSize
  • OodleLZDecoder_MemorySizeNeeded
  • OodleLZDecoder_Reset
  • OodleLZSeekTable_Flags
  • OodleLZSeekTable_Flags_Force32
  • OodleLZSeekTable_Flags_MakeRawCRCs
  • OodleLZSeekTable_Flags_None
  • OodleLZ_CheckCRC
  • OodleLZ_CheckCRC_Force32
  • OodleLZ_CheckCRC_No
  • OodleLZ_CheckCRC_Yes
  • OodleLZ_CheckSeekTableCRCs
  • OodleLZ_Compress
  • OodleLZ_CompressOptions
  • OodleLZ_CompressOptions_GetDefault
  • OodleLZ_CompressOptions_Validate
  • OodleLZ_CompressScratchMemBoundType
  • OodleLZ_CompressScratchMemBoundType_Force32
  • OodleLZ_CompressScratchMemBoundType_Typical
  • OodleLZ_CompressScratchMemBoundType_WorstCase
  • OodleLZ_CompressionLevel
  • OodleLZ_CompressionLevel_Fast
  • OodleLZ_CompressionLevel_Force32
  • OodleLZ_CompressionLevel_GetName
  • OodleLZ_CompressionLevel_HyperFast
  • OodleLZ_CompressionLevel_HyperFast1
  • OodleLZ_CompressionLevel_HyperFast2
  • OodleLZ_CompressionLevel_HyperFast3
  • OodleLZ_CompressionLevel_HyperFast4
  • OodleLZ_CompressionLevel_Invalid
  • OodleLZ_CompressionLevel_Max
  • OodleLZ_CompressionLevel_Min
  • OodleLZ_CompressionLevel_None
  • OodleLZ_CompressionLevel_Normal
  • OodleLZ_CompressionLevel_Optimal
  • OodleLZ_CompressionLevel_Optimal1
  • OodleLZ_CompressionLevel_Optimal2
  • OodleLZ_CompressionLevel_Optimal3
  • OodleLZ_CompressionLevel_Optimal4
  • OodleLZ_CompressionLevel_Optimal5
  • OodleLZ_CompressionLevel_SuperFast
  • OodleLZ_CompressionLevel_VeryFast
  • OodleLZ_Compressor
  • OodleLZ_Compressor_BitKnit
  • OodleLZ_Compressor_CanDecodeFuzzSafe
  • OodleLZ_Compressor_CanDecodeInCircularWindow
  • OodleLZ_Compressor_CanDecodeInPlace
  • OodleLZ_Compressor_CanDecodeThreadPhased
  • OodleLZ_Compressor_Count
  • OodleLZ_Compressor_Force32
  • OodleLZ_Compressor_GetName
  • OodleLZ_Compressor_Hydra
  • OodleLZ_Compressor_Invalid
  • OodleLZ_Compressor_Kraken
  • OodleLZ_Compressor_LZA
  • OodleLZ_Compressor_LZB16
  • OodleLZ_Compressor_LZBLW
  • OodleLZ_Compressor_LZH
  • OodleLZ_Compressor_LZHLW
  • OodleLZ_Compressor_LZNA
  • OodleLZ_Compressor_LZNIB
  • OodleLZ_Compressor_Leviathan
  • OodleLZ_Compressor_Mermaid
  • OodleLZ_Compressor_MustDecodeWithoutResets
  • OodleLZ_Compressor_None
  • OodleLZ_Compressor_RespectsDictionarySize
  • OodleLZ_Compressor_Selkie
  • OodleLZ_Compressor_UsesLargeWindow
  • OodleLZ_Compressor_UsesWholeBlockQuantum
  • OodleLZ_CreateSeekTable
  • OodleLZ_DecodeSome_Out
  • OodleLZ_Decode_ThreadPhase
  • OodleLZ_Decode_ThreadPhase1
  • OodleLZ_Decode_ThreadPhase2
  • OodleLZ_Decode_ThreadPhaseAll
  • OodleLZ_Decode_Unthreaded
  • OodleLZ_Decompress
  • OodleLZ_FillSeekTable
  • OodleLZ_FindSeekEntry
  • OodleLZ_FreeSeekTable
  • OodleLZ_FuzzSafe
  • OodleLZ_FuzzSafe_No
  • OodleLZ_FuzzSafe_Yes
  • OodleLZ_GetAllChunksCompressor
  • OodleLZ_GetChunkCompressor
  • OodleLZ_GetCompressScratchMemBound
  • OodleLZ_GetCompressScratchMemBoundEx
  • OodleLZ_GetCompressedBufferSizeNeeded
  • OodleLZ_GetCompressedStepForRawStep
  • OodleLZ_GetDecodeBufferSize
  • OodleLZ_GetFirstChunkCompressor
  • OodleLZ_GetInPlaceDecodeBufferSize
  • OodleLZ_GetNumSeekChunks
  • OodleLZ_GetSeekEntryPackedPos
  • OodleLZ_GetSeekTableMemorySizeNeeded
  • OodleLZ_Jobify
  • OodleLZ_Jobify_Aggressive
  • OodleLZ_Jobify_Count
  • OodleLZ_Jobify_Default
  • OodleLZ_Jobify_Disable
  • OodleLZ_Jobify_Force32
  • OodleLZ_Jobify_GetName
  • OodleLZ_Jobify_Normal
  • OodleLZ_MakeSeekChunkLen
  • OodleLZ_PackedRawOverlap
  • OodleLZ_PackedRawOverlap_Force32
  • OodleLZ_PackedRawOverlap_No
  • OodleLZ_PackedRawOverlap_Yes
  • OodleLZ_Profile
  • OodleLZ_Profile_Force32
  • OodleLZ_Profile_Main
  • OodleLZ_Profile_Reduced
  • OodleLZ_SeekTable
  • OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
  • OodleLZ_Verbosity
  • OodleLZ_Verbosity_Force32
  • OodleLZ_Verbosity_Lots
  • OodleLZ_Verbosity_Minimal
  • OodleLZ_Verbosity_None
  • OodleLZ_Verbosity_Some
  • OodleNet_Plugins_SetAllocators
  • OodleNet_Plugins_SetAssertion
  • OodleNet_Plugins_SetJobSystem
  • OodleNet_Plugins_SetJobSystemAndCount
  • OodleNet_Plugins_SetPrintf
  • OodleNetwork1TCP_Decode
  • OodleNetwork1TCP_Encode
  • OodleNetwork1TCP_State
  • OodleNetwork1TCP_State_InitAsCopy
  • OodleNetwork1TCP_State_Reset
  • OodleNetwork1TCP_State_Size
  • OodleNetwork1TCP_Train
  • OodleNetwork1UDP_Decode
  • OodleNetwork1UDP_Encode
  • OodleNetwork1UDP_State
  • OodleNetwork1UDP_StateCompacted
  • OodleNetwork1UDP_StateCompacted_MaxSize
  • OodleNetwork1UDP_State_Compact
  • OodleNetwork1UDP_State_Compact_ForVersion
  • OodleNetwork1UDP_State_Size
  • OodleNetwork1UDP_State_Uncompact
  • OodleNetwork1UDP_State_Uncompact_ForVersion
  • OodleNetwork1UDP_Train
  • OodleNetwork1_CompressedBufferSizeNeeded
  • OodleNetwork1_SelectDictionaryFromPackets
  • OodleNetwork1_SelectDictionaryFromPackets_Trials
  • OodleNetwork1_SelectDictionarySupported
  • OodleNetwork1_Shared
  • OodleNetwork1_Shared_SetWindow
  • OodleNetwork1_Shared_Size
  • OodleNetworkVersion
  • OodleX Debug aids
  • OodleX LZ compression
  • OodleX Memory Allocators
  • OodleX Startup and Shutdown
  • OodleX Utils
  • OodleX async handle operations
  • OodleX low level async io
  • OodleX threading util
  • OodleXAPI_LZ_Async
  • OodleXAsyncSelect
  • OodleXAsyncSelect_All
  • OodleXAsyncSelect_Force32
  • OodleXAsyncSelect_Full
  • OodleXAsyncSelect_NoFlagsMask
  • OodleXAsyncSelect_None
  • OodleXAsyncSelect_Wide
  • OodleXAsyncSelect_Workers
  • OodleXConfigValues
  • OodleXCopyFileFlags
  • OodleXCopyFileFlags_Default
  • OodleXCopyFileFlags_DontOverwriteExisting
  • OodleXCopyFileFlags_Force32
  • OodleXCopyFileFlags_Mask
  • OodleXCopyFileFlags_Overwrite
  • OodleXCopyFileFlags_OverwriteOnlyIfDifferentSize
  • OodleXCopyFileFlags_OverwriteOnlyIfNewer
  • OodleXCopyFileFlags_OverwriteOnlyIfNewerOrDifferentSize
  • OodleXDecompressCallback_WriteFile_Data_Init
  • OodleXError
  • OodleXError_Alignment
  • OodleXError_BadParameters
  • OodleXError_Close
  • OodleXError_Compressor
  • OodleXError_Corrupt
  • OodleXError_Count
  • OodleXError_FileNotFound
  • OodleXError_Force32
  • OodleXError_InvalidHandle
  • OodleXError_Malloc
  • OodleXError_NoAccess
  • OodleXError_Ok
  • OodleXError_PreviousAsyncFailed
  • OodleXError_UnexpectedEOF
  • OodleXError_Unknown
  • OodleXFileInfo
  • OodleXFileMode
  • OodleXFileMode_Force32
  • OodleXFileMode_Invalid
  • OodleXFileMode_Read
  • OodleXFileMode_ReadWrite
  • OodleXFileMode_Write
  • OodleXFileMode_WriteCreate
  • OodleXFileOpenFlags
  • OodleXFileOpenFlags_Buffered
  • OodleXFileOpenFlags_Default
  • OodleXFileOpenFlags_Force32
  • OodleXFileOpenFlags_NotBuffered
  • OodleXFileOpenFlags_WriteCreateDontStomp
  • OodleXFileOpsVTable
  • OodleXFree
  • OodleXFreeBig
  • OodleXFreeSized
  • OodleXFree_IOAligned
  • OodleXHandle
  • OodleXHandleAutoDelete
  • OodleXHandleAutoDelete_Force32
  • OodleXHandleAutoDelete_No
  • OodleXHandleAutoDelete_Yes
  • OodleXHandleCountdown_Alloc
  • OodleXHandleCountdown_Decrement
  • OodleXHandleDeleteIfDone
  • OodleXHandleDeleteIfDone_Force32
  • OodleXHandleDeleteIfDone_No
  • OodleXHandleDeleteIfDone_Yes
  • OodleXHandleEvent_Alloc
  • OodleXHandleEvent_SetDone
  • OodleXHandleEvent_SetError
  • OodleXHandleKickDelayed
  • OodleXHandleKickDelayed_Force32
  • OodleXHandleKickDelayed_No
  • OodleXHandleKickDelayed_Yes
  • OodleXIOQFile
  • OodleXIOQ_ClearError
  • OodleXIOQ_CloseFileRename_Async
  • OodleXIOQ_CloseFile_Async
  • OodleXIOQ_CopyFile_Async
  • OodleXIOQ_CopyFile_AsyncAndWait
  • OodleXIOQ_Delete_Async
  • OodleXIOQ_Delete_AsyncAndWait
  • OodleXIOQ_Fence_Async
  • OodleXIOQ_ForceWriteable_Async
  • OodleXIOQ_FreeBufferIOAligned_Async
  • OodleXIOQ_GetErrorDetails
  • OodleXIOQ_GetErrorEnum
  • OodleXIOQ_GetFileSize_AsyncAndWait
  • OodleXIOQ_GetInfo
  • OodleXIOQ_GetInfoByName_Async
  • OodleXIOQ_GetInfoByName_AsyncAndWait
  • OodleXIOQ_GetInfoByName_GetResult
  • OodleXIOQ_GetLastError
  • OodleXIOQ_GetLastPendingOpOnFile
  • OodleXIOQ_GetName
  • OodleXIOQ_GetOSHandle
  • OodleXIOQ_GetStatus
  • OodleXIOQ_KickAnyDelayed
  • OodleXIOQ_LogError
  • OodleXIOQ_LogLastError
  • OodleXIOQ_MakeAllDirs_Async
  • OodleXIOQ_MakeAllDirs_AsyncAndWait
  • OodleXIOQ_MakeDir_Async
  • OodleXIOQ_NameIsDir_AsyncAndWait
  • OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
  • OodleXIOQ_OpenAndReadMallocWholeFile_Async
  • OodleXIOQ_OpenAndRead_Async
  • OodleXIOQ_OpenForRead_Async
  • OodleXIOQ_OpenForWriteCreate_Async
  • OodleXIOQ_OpenForWriteTempName_Async
  • OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
  • OodleXIOQ_OpenWriteWholeFileClose_Async
  • OodleXIOQ_ReadMallocWholeFile_Async
  • OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
  • OodleXIOQ_ReadMallocWholeFile_GetResult
  • OodleXIOQ_ReadUnalignedAdjustPointer_Async
  • OodleXIOQ_Read_Async
  • OodleXIOQ_Rename_Async
  • OodleXIOQ_Rename_AsyncAndWait
  • OodleXIOQ_ReserveFileSizeForWrite_Async
  • OodleXIOQ_SetFileSize_Async
  • OodleXIOQ_SetInfoByName_Async
  • OodleXIOQ_SetInfoByName_AsyncAndWait
  • OodleXIOQ_SetVTable
  • OodleXIOQ_Wait
  • OodleXIOQ_WaitDoneAllPending
  • OodleXIOQ_Wait_GetInfo
  • OodleXIOQ_WriteWholeFile_AsyncAndWait
  • OodleXIOQ_Write_Async
  • OodleXInitOptions
  • OodleXLZ_Compress_Async
  • OodleXLZ_Compress_AsyncAndWait
  • OodleXLZ_Compress_Wait_GetResult
  • OodleXLZ_Decompress_MakeSeekTable_Wide_Async
  • OodleXLZ_Decompress_Narrow_Async
  • OodleXLZ_Decompress_ThreadPhased_Narrow_Async
  • OodleXLZ_Decompress_Wide_Async
  • OodleXLZ_ReadAndDecompress_Stream_Async
  • OodleXLZ_ReadAndDecompress_Wide_Async
  • OodleXLogCallbackRet
  • OodleXLogCallbackRetRet
  • OodleXLogCallbackRetRet_Continue
  • OodleXLogCallbackRetRet_Force32
  • OodleXLogCallbackRetRet_Terminate
  • OodleXLog_Flush
  • OodleXLog_GetCallback
  • OodleXLog_GetEcho
  • OodleXLog_GetVerboseLevel
  • OodleXLog_Printf
  • OodleXLog_PrintfError
  • OodleXLog_SetCallback
  • OodleXLog_SetEcho
  • OodleXLog_SetState
  • OodleXLog_SetVerboseLevel
  • OodleXLog_StateFlags
  • OodleXLog_VerboseLevel
  • OodleXLog_Verbose_Force32
  • OodleXLog_Verbose_Lots
  • OodleXLog_Verbose_Minimal
  • OodleXLog_Verbose_None
  • OodleXLog_Verbose_Some
  • OodleXMalloc
  • OodleXMallocAligned
  • OodleXMallocBig
  • OodleXMallocBigAlignment
  • OodleXMallocFailedHandler
  • OodleXMallocVTable
  • OodleXMalloc_GetVTable_Clib
  • OodleXMalloc_GetVTable_OS
  • OodleXMalloc_IOAligned
  • OodleXMalloc_InstallVTable
  • OodleXMalloc_OS_Options
  • OodleXMalloc_OS_Options_Count
  • OodleXMalloc_OS_Options_Force32
  • OodleXMalloc_OS_Options_GuardBig
  • OodleXMalloc_OS_Options_GuardBoth
  • OodleXMalloc_OS_Options_GuardFrees
  • OodleXMalloc_OS_Options_None
  • OodleXMalloc_SetFailedHandler
  • OodleXMalloc_ValidatePointer
  • OodleXOSFile
  • OodleXOSFileListing
  • OodleXPriority
  • OodleXPriority_Default
  • OodleXPriority_Force32
  • OodleXPriority_NoPopOnWait
  • OodleXPriority_Normal
  • OodleXStatus
  • OodleXStatus_Count
  • OodleXStatus_Done
  • OodleXStatus_Error
  • OodleXStatus_Force32
  • OodleXStatus_Invalid
  • OodleXStatus_Pending
  • OodleXUtil_ConvertUTF16ToUTF8
  • OodleXUtil_ConvertUTF8ToUTF16
  • OodleX_CombinePaths
  • OodleX_CorePlugin_RunJob
  • OodleX_CorePlugin_WaitJob
  • OodleX_CreateThread
  • OodleX_GetAvailableAsyncSelect
  • OodleX_GetConfigValues
  • OodleX_GetDefaultFileOps
  • OodleX_GetExtensionKey
  • OodleX_GetNumWorkerThreads
  • OodleX_GetOSCwd
  • OodleX_GetOSFileOps
  • OodleX_GetSeconds
  • OodleX_GetStatus
  • OodleX_IOAlignDownS32
  • OodleX_IOAlignDownS64
  • OodleX_IOAlignDownSINTa
  • OodleX_IOAlignUpS32
  • OodleX_IOAlignUpS64
  • OodleX_IOAlignUpSINTa
  • OodleX_Init
  • OodleX_Init_Default
  • OodleX_Init_GetDefaults
  • OodleX_Init_GetDefaults_DebugSystems
  • OodleX_Init_GetDefaults_DebugSystems_Force32
  • OodleX_Init_GetDefaults_DebugSystems_No
  • OodleX_Init_GetDefaults_DebugSystems_Yes
  • OodleX_Init_GetDefaults_Minimal
  • OodleX_Init_GetDefaults_Threads
  • OodleX_Init_GetDefaults_Threads_Force32
  • OodleX_Init_GetDefaults_Threads_No
  • OodleX_Init_GetDefaults_Threads_Yes
  • OodleX_Init_NoThreads
  • OodleX_Init_ThreadProfilerInit
  • OodleX_LogSystemInfo
  • OodleX_MakeExtensionKey
  • OodleX_PrefixOSCwd
  • OodleX_ReleaseThreadTLS
  • OodleX_S64_to_SINTa_check
  • OodleX_Semaphore
  • OodleX_Semaphore_Post
  • OodleX_Semaphore_Wait
  • OodleX_SetConfigValues
  • OodleX_SetDefaultFileOps
  • OodleX_SetHandleAutoDelete
  • OodleX_Shutdown
  • OodleX_Shutdown_DebugBreakOnLeaks
  • OodleX_Shutdown_DebugBreakOnLeaks_Force32
  • OodleX_Shutdown_DebugBreakOnLeaks_No
  • OodleX_Shutdown_DebugBreakOnLeaks_Yes
  • OodleX_Shutdown_LogLeaks
  • OodleX_Shutdown_LogLeaks_Force32
  • OodleX_Shutdown_LogLeaks_No
  • OodleX_Shutdown_LogLeaks_Yes
  • OodleX_Shutdown_NoThreads
  • OodleX_ThreadFunc
  • OodleX_Wait
  • OodleX_WaitAll
  • OodleX_WaitAndDestroyThread
  • OodleX_WaitDoneAllPending
  • Oodle_CheckVersion
  • Oodle_GetConfigValues
  • Oodle_LogHeader
  • Oodle_SetConfigValues
  • Oodle_SetUsageWarnings
  • Oodle_UsageWarnings
  • Oodle_UsageWarnings_Disabled
  • Oodle_UsageWarnings_Enabled
  • Oodle_UsageWarnings_Force32
  • Third Party License Notices
  • Tips for benchmarking a compressor
  • Welcome to Oodle
  • closeH
  • compBufUsed
  • compressor
  • curQuantumCompLen
  • curQuantumRawLen
  • decodedCount
  • delete_file
  • dictionarySize
  • doCloseFile
  • example_lz : Example demonstrating LZ compression and decompression
  • example_lz_chart : Example that makes a chart of OodleLZ options
  • example_lz_noallocs : Example demonstrating Oodle compression with no allocations
  • example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
  • example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
  • example_lz_threadphased : Example of 2-thread ThreadPhased decoding
  • example_network_client : Example with simple network client support
  • example_packet : Example demonstrating network packet compression
  • farMatchMinLen
  • farMatchOffsetLog2
  • file
  • fileops_data
  • flags
  • force_writeable
  • get_error_enum
  • get_error_string
  • get_filename_info
  • get_last_error
  • jobify
  • jobifyUserPtr
  • lastWriteH
  • listdir_close
  • listdir_next
  • listdir_open
  • m_OodleIOQStream_MaxReadSize
  • m_OodleIOQStream_MinReadSize
  • m_OodleIOQStream_OffsetAlignment
  • m_OodleInit_BreakOnLogError
  • m_OodleInit_DebugAllocator
  • m_OodleInit_FuzzTest
  • m_OodleInit_IOQ
  • m_OodleInit_IOQ_BreakOnError
  • m_OodleInit_IOQ_CheckAlignment
  • m_OodleInit_IOQ_Log
  • m_OodleInit_IOQ_Threaded
  • m_OodleInit_LeakTrack
  • m_OodleInit_Log
  • m_OodleInit_Log_FileName
  • m_OodleInit_Log_FlushEachWrite
  • m_OodleInit_Log_Header
  • m_OodleInit_SimpleProf
  • m_OodleInit_StackTrace
  • m_OodleInit_Telemetry
  • m_OodleInit_Telemetry_Context
  • m_OodleInit_ThreadLog
  • m_OodleInit_ThreadProfiler_funcptr
  • m_OodleInit_Workers
  • m_OodleInit_Workers_Count
  • m_OodleLZ_BackwardsCompatible_MajorVersion
  • m_OodleLZ_Decoder_Max_Stack_Size
  • m_OodleLZ_LW_LRM_hashLength
  • m_OodleLZ_LW_LRM_jumpbits
  • m_OodleLZ_LW_LRM_step
  • m_OodleLZ_Small_Buffer_LZ_Fallback_Size_Unused
  • m_Oodle_DefaultIOBufferSize
  • m_Oodle_DefaultWriteReserveSize
  • m_Oodle_MaxSingleIOSize
  • m_Oodle_OSFileOpen_Default_Read_Buffered
  • m_Oodle_OSFileOpen_Default_Write_Buffered
  • m_Oodle_PathsCaseSensitive
  • m_Oodle_very_long_wait_seconds
  • m_bigAlignment
  • m_context
  • m_deprecated_Desired_Parallel_BranchFactor
  • m_num_handles_log2
  • m_oodle_header_version:
    • OodleConfigValues
    • OodleXConfigValues
  • m_pBaseVTable
  • m_pFree
  • m_pFreeBig
  • m_pFreeSized
  • m_pMalloc
  • m_pMallocAligned
  • m_pMallocBig
  • m_pValidatePointer
  • makeLongRangeMatcher
  • make_dir
  • matchTableSizeLog2
  • maxLocalDictionarySize
  • minMatchLen
  • modTime
  • numSeekChunks
  • pad
  • profile
  • rawCRCs
  • rename_file
  • reserved
  • seekChunkCompLens
  • seekChunkLen:
    • OodleLZ_CompressOptions
    • OodleLZ_SeekTable
  • seekChunkReset
  • seekChunksIndependent
  • sendQuantumCRCs
  • set_filename_info
  • size
  • spaceSpeedTradeoffBytes
  • sync_close
  • sync_get_file_info
  • sync_open
  • sync_read
  • sync_set_file_size
  • sync_write
  • t_OodleFPVoidVoid
  • t_OodleFPVoidVoidStar
  • t_fp_OodleCore_Plugin_DisplayAssertion
  • t_fp_OodleCore_Plugin_Free
  • t_fp_OodleCore_Plugin_MallocAligned
  • t_fp_OodleCore_Plugin_Printf
  • t_fp_OodleCore_Plugin_RunJob
  • t_fp_OodleCore_Plugin_WaitJob
  • t_fp_OodleNet_Plugin_DisplayAssertion
  • t_fp_OodleNet_Plugin_Free
  • t_fp_OodleNet_Plugin_MallocAligned
  • t_fp_OodleNet_Plugin_Printf
  • t_fp_OodleNet_Plugin_RunJob
  • t_fp_OodleNet_Plugin_WaitJob
  • t_fp_Oodle_Job
  • totalCompLen
  • totalRawLen
  • unused_was_maxHuffmansPerChunk
  • unused_was_verbosity
  • written
About Oodle on Nintendo Switch
Navigation
 About Oodle on PS4
 About Oodle on Platforms
 About Oodle on Mac
 Welcome to Oodle
 Change Log

Oodle for Switch is provided as a lib.

lib/liboo2coreswitch.a
lib/liboo2extswitch.a

The debug build of the Oodle lib is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems.


OodleX file IO on Switch uses stdio.

OodleX Malloc on Switch uses stdlib malloc.

NOTE to run the examples on Switch, you must mount some directory for them to work on, and change the paths used to be relative to that mount.

You could add :

	nn::fs::MountHostRoot();

then run "example_lz_chart c:test_file"

In example_lz : Example demonstrating LZ compression and decompression these would need to be changed to be relative to the mount point with an absolute path :

static const char * in_name = "oodle_example_input_file";
static const char * out_name = "oodle_example_output_file";

 
About Oodle on PS4About Oodle on PlatformsAbout Oodle on Mac

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_NoThreads
Navigation
 OodleX_Shutdown
 OodleX_Shutdown_NoThreads
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleX_Init_NoThreads( OO_U32 oodle_header_version,
                               const OodleXInitOptions * pOptions );
Discussion
Initialize Oodle with no threads and minimal systems
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
pOptions  options for Init; must not be NULL; use OodleX_Init_GetDefaults_Minimal to fill out
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

You must call OodleX_Init or OodleX_Init_NoThreads before any other Oodle function that you expect to work.

Pair with OodleX_Shutdown_NoThreads.

This function does not enable the Oodle IOQ or WorkMgr. No async jobs or IO will work.

All memory->memory compressors will work.

Pair with OodleX_Shutdown_NoThreads
 

OodleX_ShutdownOodleX Startup and ShutdownOodleX_Shutdown_NoThreads

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFree_IOAligned
Navigation
 OodleXMalloc_IOAligned
 OodleXMalloc_GetVTable_Clib
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXFree_IOAligned( void * ptr );
Discussion
Free a pointer allocated with OodleXMalloc_IOAligned
Parameters
ptr  pointer to free

 
OodleXMalloc_IOAlignedOodleX Memory AllocatorsOodleXMalloc_GetVTable_Clib

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMallocBigAlignment
Navigation
 OodleXFreeSized
 OodleXMallocBig
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleXMallocBigAlignment( );
Discussion
returns the alignment of OodleXMallocBig pointers
Return Value
return  the alignment of OodleXMallocBig pointers
Discussion

Should be >= OODLEX_IO_MAX_ALIGNMENT
 

OodleXFreeSizedOodleX Memory AllocatorsOodleXMallocBig

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_CreateThread
Navigation
 OodleX_Semaphore_Wait
 OodleX_WaitAndDestroyThread
 Welcome to Oodle
 Change Log
// Function prototype:
OodleX_Thread OodleX_CreateThread( OodleX_ThreadFunc * func,
                                   void * userdata );
Discussion
Start a thread running threadfunc
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX_Semaphore_WaitOodleX threading utilOodleX_WaitAndDestroyThread

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleDeleteIfDone
Navigation
 OodleXHandleKickDelayed
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXHandleDeleteIfDone
{
    OodleXHandleDeleteIfDone_No = 0,
    OodleXHandleDeleteIfDone_Yes = 1,
    OodleXHandleDeleteIfDone_Force32 = 0x40000000
};
Discussion
Pass OodleXHandleDeleteIfDone_Yes to handle status checks to delete the handle if it's done. This is the main way to free an OodleXHandle
Enumerants
OodleXHandleDeleteIfDone_No  (default) do not delete the handle
OodleXHandleDeleteIfDone_Yes  delete the handle if it's Status is Done or Error
OodleXHandleDeleteIfDone_Force32 

 
OodleXHandleKickDelayedOodleX async handle operationsOodleX_GetStatus

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_ValidatePointer
Navigation
 OodleXFreeBig
 OodleXMalloc_IOAligned
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXMalloc_ValidatePointer( const void * ptr,
                                      OO_SINTa bytes );
Discussion
debug check if a pointer is a valid malloc
Parameters
ptr  pointer to validate
bytes  size of allocation if known; -1 if not
Return Value
return  true if the malloc headers are all okay
Discussion

Should work on OodleXMalloc and OodleXMallocBig pointers. Bytes can be -1 if unknown, but there will be less validation checks. ValidatePointer is most useful if the OodleXMalloc debug thunk layer is installed in OodleX_Init.
 

OodleXFreeBigOodleX Memory AllocatorsOodleXMalloc_IOAligned

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Frequently Asked Questions
Navigation
 Welcome to Oodle
 Change Log

Oodle FAQ

Frequently Asked Questions :

  •  FAQ: OodleLZ_Decompress is failing how do I diagnose it?
  •  FAQ: How do I use Oodle with no allocator?
  •  FAQ: How do I limit the encoder memory use?
  •  FAQ: How much memory do the Oodle compressors use ?
  •  FAQ: What are the Oodle deprecated compressors ?
  •  FAQ: Do new Oodle versions break data compatibility ?
  •  FAQ: How does OodleLZ compare with other compressors ?
  •  FAQ: Which OodleLZ should I use?
  •  FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
  •  FAQ: How do I decompress to graphics memory quickly?
  •  FAQ: How do I get the Oodle logs?
  •  FAQ: I write a file with IOQ but the contents are garbage?
  •  FAQ: I ran out of OodleHandle table slots; what do I do?
  •  FAQ: What is SINTa? How do I load files bigger than 2 GB?
  •  FAQ: My Files aren't loading right and I can't track it down

 
  FAQ: OodleLZ_Decompress is failing how do I diagnose it?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_JOB_MAX_DEPENDENCIES
Navigation
 OODLE_MALLOC_MINIMUM_ALIGNMENT
 OODLE_JOB_NULL_HANDLE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_JOB_MAX_DEPENDENCIES (4)
Discussion
Maximum number of dependencies Oodle will ever pass to a RunJob callback
 
OODLE_MALLOC_MINIMUM_ALIGNMENTCore BaseOODLE_JOB_NULL_HANDLE

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_RunJob
Navigation
 t_fp_OodleCore_Plugin_Free
 t_fp_OodleCore_Plugin_WaitJob
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC OO_U64( OODLE_CALLBACK t_fp_OodleCore_Plugin_RunJob )( t_fp_Oodle_Job * fp_job,
                  void * job_data,
                  OO_U64 * dependencies,
                  int num_dependencies,
                  void * user_ptr );
Discussion
Function pointer type for OodleCore_Plugins_SetJobSystem
Parameters
dependencies  array of handles of other pending jobs. All guaranteed to be nonzero.
num_dependencies  number of dependencies. Guaranteed to be no more than OODLE_JOB_MAX_DEPENDENCIES.
user_ptr  is passed through from the OodleLZ_CompressOptions.
Return Value
return  handle to the async job, or 0 if it was run synchronously
Discussion

RunJob will call fp_job(job_data)

it may be done on a thread, or it may run the function synchronously and return 0, indicating the job is already done. The returned OO_U64 is a handle passed to WaitJob, unless it is 0, in which case WaitJob won't get called.

fp_job should not run until all the dependencies are done. This function should not delete the dependencies.

RunJob must be callable from within an Oodle Job, i.e. jobs may spawn their own sub-jobs directly. However, the matching WaitJob calls will only ever occur on the thread that called the internally threaded Oodle API function.

See About Oodle Job Threading Plugins
 

t_fp_OodleCore_Plugin_FreeCore pluginst_fp_OodleCore_Plugin_WaitJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Decompress_Wide_Async
Navigation
 OodleXLZ_ReadAndDecompress_Wide_Async
 OodleXLZ_Decompress_MakeSeekTable_Wide_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_Decompress_Wide_Async( OO_U32 asyncSelect,
                                             const OodleLZ_SeekTable * seekTable,
                                             const void * packedDataPtr,
                                             OO_SINTa packedLen,
                                             void * rawArray,
                                             OO_SINTa rawArrayLen,
                                             OodleLZ_FuzzSafe fuzzSafe OODEFAULT( OodleLZ_FuzzSafe_No ),
                                             OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                                             OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                                             void * decBufBase OODEFAULT( NULL ),
                                             OO_SINTa decBufSize OODEFAULT( 0 ),
                                             OodleLZ_PackedRawOverlap packedRawOverlap OODEFAULT( OodleLZ_PackedRawOverlap_No ),
                                             OodleXIOQFile writeToFile OODEFAULT( 0 ),
                                             OO_S64 writeToFileStartPos OODEFAULT( 0 ),
                                             OodleXHandle * pWriteHandleGroup OODEFAULT( 0 ),
                                             OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                             const OodleXHandle * dependencies OODEFAULT( NULL ),
                                             OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an async LZ decompress, possibly write raw data
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run
seekTable  seek locations as created by OodleLZ_CreateSeekTable
packedDataPtr  pointer to LZ compressed data
packedLen  compressed data length
rawArray  pointer to memory filled with decompressed data
rawArrayLen  length of decompressed data
checkCRC  if OodleLZ_CheckCRC_Yes, the decompressor checks the crc to ensure data integrity
verbosity  (optional) if not OodleLZ_Verbosity_None, will log some information
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder.
decBufSize  (optional) size of circular buffer starting at decBufBase
packedRawOverlap  (optional) if OodleLZ_PackedRawOverlap_Yes, the compressed data is in the same memory array as the output raw data
writeToFile  (optional) OodleXIOQFile to write raw data to
writeToFileStartPos  (optional) file position where writeToFile should start (must be OODLEX_IO_MAX_ALIGNMENT aligned)
pWriteHandleGroup  (optional) if writeToFile is given, this is filled with an OodleAsyncGroup OodleXHandle containing all the file IO operations
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  OodleXHandle to the operation, or OodleXHandle_Null for invalid arguments
Discussion

Same as OodleXLZ_ReadAndDecompress_Wide_Async, except this API doesn't include the option to read the packed data, it must be already fully loaded.


 

OodleXLZ_ReadAndDecompress_Wide_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Decompress_MakeSeekTable_Wide_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_State_Reset
Navigation
 OodleNetwork1_Shared_SetWindow
 OodleNetwork1TCP_State_InitAsCopy
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNetwork1TCP_State_Reset( OodleNetwork1TCP_State * state );
Discussion
Initialize a OodleNetwork1TCP_State
Parameters
state  OodleNetwork1TCP_State to initialize
Discussion

Resets state to a null state.

Generally it is better to make a trained initial state with OodleNetwork1TCP_Train and then use OodleNetwork1TCP_State_InitAsCopy.
 

OodleNetwork1_Shared_SetWindowOodleAPI_OodleNetwork1OodleNetwork1TCP_State_InitAsCopy

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Getting Started with Oodle Network
Navigation
 Welcome to Oodle
 Change Log

Oodle Network compresses packets for bandwidth reduction and improved player experience in networked games. It works with both TCP and UDP.

To compress packages or downloadable content, see Getting Started with Oodle LZ Data Compression

If you are using the Unreal Engine, ask us about our Unreal integration.

Oodle Network does most of its work in an offline training phase. You capture a large sample of packets. It uses that capture to build a model of your data and learns how to compress it. That model is saved to disk, and will be shipped with your game.

The runtime component loads the saved model and uses it to compress packets on the fly.

The trained model is what makes it work even on UDP packets, and even on very small packets.

The compression of packets is zero-latency (no extra buffering is introduced). UDP mode compresses packets independently. TCP mode learns from the previous data sent over each connection.

example_network_client : Example with simple network client support provides a simple example of how you would use a previously trained model to compress packets in your shipping game.

To evalute Oodle for network packet compression, you need to first get a capture of your typical network stream to test on.

You should try to capture packets from a realistic game run (perhaps from your QA department playing the game). The captured packets should be without any encryption or other compression algorithms applied. You may continue to use your bit-packing or delta scheme. Try to capture 100 MB of packet data.

The more realistic and varied your packet capture is, the more the test will reflect real world performance. The capture should not include network protocol headers.

The packet capture format that example_packet loads is :

packet.bin :
U32 [LE] : numbers of channels (num_channels)
repeatedly :
{
	U32 [LE] : channel index in [0,num_channels-1]
	U32 [LE] : number of bytes of data in this packet (num_bytes)
	U8 * num_bytes : payload of this packet
}
// (for UDP set num_channels to 1 and all the channel indices to 0)

Once you have a capture, you can run example_packet : Example demonstrating network packet compression to test Oodle's compressors on it. Or you can send to us, and we'll run the tests for you. Contact oodle@radgametools.com for details.

NOTE : captured packets should have any existing compression and encryption disabled. Packets should not contain IP headers. Usually it is best to log the packet data from your network code, don't try to use an external packet logger. If possible, log packets before splitting them into MTU units. The goal is to log the raw packet payloads.

See Capturing Training data for OodleNetwork.

What example_packet does is load the captured packets, and splits them into a training set and a testing set. It then trains a model (TestOodleNetwork_SelectDictionaryAndTrain), and saves it to disk (OodleNetwork1_Compressor_WriteToFile). It then loads the model from disk, as the game runtime would (OodleNetwork1_Compressor_LoadFromFileData), and then tests the performance of the model on the test holdout packets (TestOodleNetwork1UDPPacketCoder_Transmission).

The APIs used for the actual runtime packet compression are OodleNetwork1UDP_Encode and OodleNetwork1UDP_Decode . (for UDP, similar variants for TCP; toggle testing UDP or TCP with a #define in example_packet).

See example_packet : Example demonstrating network packet compression for details. It provides a model trainer which you can either use directly, or use as a starting point for incorporating your own variant in your own tools.

See example_network_client : Example with simple network client support for example of how to use Oodle Network in your shipping runtime.

For more, see About OodleNetwork1, also Capturing Training data for OodleNetwork and Forming Packets for Maximum Compression

For a guide to how to build with the Oodle library on your platform, see About Oodle on Platforms. For information about the Core vs Ext libs see Oodle2 Core vs Oodle2 Ext.
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State_Size
Navigation
 OodleNetwork1UDP_Train
 OodleNetwork1UDP_Encode
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1UDP_State_Size( void );
Discussion
Returns the size of memory required for an OodleNetwork1UDP_State object
Discussion
Shared and State are allocated with malloc( Size() )


 

OodleNetwork1UDP_TrainOodleAPI_OodleNetwork1OodleNetwork1UDP_Encode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleDecompressCallbackRet
Navigation
 OodleLZ_Profile
 OodleLZ_CompressionLevel
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleDecompressCallbackRet
{
    OodleDecompressCallbackRet_Continue = 0,
    OodleDecompressCallbackRet_Cancel = 1,
    OodleDecompressCallbackRet_Invalid = 2,
    OodleDecompressCallbackRet_Force32 = 0x40000000
};
Discussion
Return value for OodleDecompressCallback return OodleDecompressCallbackRet_Cancel to abort the in-progress decompression
 
OodleLZ_ProfileOodleAPI_LZ_CompressorsOodleLZ_CompressionLevel

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_QUANTUM_LEN
Navigation
 OODLELZ_BLOCK_MAX_COMPLEN
 OODLELZ_FAILED
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_QUANTUM_LEN (1<<14)
Discussion
Minimum decompression quantum (for old legacy codecs only)
Discussion
Deprecated.

The new sea monster family of compressors use a whole block quantum (OODLELZ_BLOCK_LEN). Check OodleLZ_Compressor_UsesWholeBlockQuantum
 

OODLELZ_BLOCK_MAX_COMPLENOodleAPI_LZ_CompressorsOODLELZ_FAILED

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Shutdown_NoThreads
Navigation
 OodleX_Init_NoThreads
 OodleX_GetConfigValues
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_Shutdown_NoThreads( const char * threadProfileLogName OODEFAULT( NULL ),
                                OodleX_Shutdown_LogLeaks logLeaks OODEFAULT( OodleX_Shutdown_LogLeaks_Yes ),
                                OO_U64 allocStartCounter OODEFAULT( 0 ),
                                OodleX_Shutdown_DebugBreakOnLeaks debugBreakOnLeaks OODEFAULT( OodleX_Shutdown_DebugBreakOnLeaks_No ) );
Discussion
Shut down Oodle at app exit time.
Parameters
threadProfileLogName  (optional) if not NULL, and the ThreadProfiler is enabled, writes the threadprofiler output to this file name
logLeaks  (optional) if true and the LeakTracker is enabled, logs any leaks or memory or handles
allocStartCounter  (optional) initial counter for the LeakTrack log
debugBreakOnLeaks  (optional) if there are any leaks, do a debug break
Discussion

Pair with OodleX_Init_NoThreads. No Oodle functions should be called after Shutdown.

Call Shutdown from the same thread that called Init.

Do not shutdown Oodle then init again. Only call Init and Shutdown once per run.
 

OodleX_Init_NoThreadsOodleX Startup and ShutdownOodleX_GetConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_GetVTable_Clib
Navigation
 OodleXFree_IOAligned
 OodleXMalloc_GetVTable_OS
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleXMallocVTable * OodleXMalloc_GetVTable_Clib( OodleXMalloc_OS_Options options );
Discussion
get an OodleMalloc VTable that contains allocators based on the std clib malloc/free
 
OodleXFree_IOAlignedOodleX Memory AllocatorsOodleXMalloc_GetVTable_OS

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_PrefixOSCwd
Navigation
 OodleX_GetOSCwd
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_PrefixOSCwd( char * addTo,
                         OO_S32 addToSize );
Discussion
Make a relative path absolute by prefixing the current dir
Parameters
addTo  string which will be modified
addToSize  total size of addTo (not the strlen)
Discussion

If addTo is an absolute path already it is not changed. If it is relative, the OS cwd is retrieved with OodleX_GetOSCwd and then prefixed in front of addTo.
 

OodleX_GetOSCwdOodleX UtilsAbout Oodle Ext

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle2 Network API Documentation
Navigation
 Welcome to Oodle
 Change Log
  •  Oodle Network compression
    •  OodleAPI_OodleNetwork1
      •  About OodleNetwork1
      •  Defines
        •  OODLENETWORK1_MAX_DICTIONARY_SIZE
        •  OODLENETWORK1_HASH_BITS_DEFAULT
        •  OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
      •  Structures
        •  OodleNetwork1_Shared
        •  OodleNetwork1TCP_State
        •  OodleNetwork1UDP_State
        •  OodleNetwork1UDP_StateCompacted
      •  Functions
        •  OodleNetwork1_Shared_Size
        •  OodleNetwork1TCP_State_Size
        •  OodleNetwork1_CompressedBufferSizeNeeded
        •  OodleNetwork1_Shared_SetWindow
        •  OodleNetwork1TCP_State_Reset
        •  OodleNetwork1TCP_State_InitAsCopy
        •  OodleNetwork1TCP_Train
        •  OodleNetwork1TCP_Encode
        •  OodleNetwork1TCP_Decode
        •  OodleNetwork1UDP_Train
        •  OodleNetwork1UDP_State_Size
        •  OodleNetwork1UDP_Encode
        •  OodleNetwork1UDP_Decode
        •  OodleNetwork1UDP_StateCompacted_MaxSize
        •  OodleNetwork1UDP_State_Compact_ForVersion
        •  OodleNetwork1UDP_State_Compact
        •  OodleNetwork1UDP_State_Uncompact_ForVersion
        •  OodleNetwork1UDP_State_Uncompact
        •  OodleNetwork1_SelectDictionarySupported
        •  OodleNetwork1_SelectDictionaryFromPackets
        •  OodleNetwork1_SelectDictionaryFromPackets_Trials
    •  About Oodle Network Compression
      •  Capturing Training data for OodleNetwork
      •  Forming Packets for Maximum Compression
  •  Network plugins
    •  Functions
      •  OodleNet_Plugins_SetAllocators
      •  OodleNet_Plugins_SetJobSystem
      •  OodleNet_Plugins_SetJobSystemAndCount
      •  OodleNet_Plugins_SetPrintf
      •  OodleNet_Plugins_SetAssertion
    •  Typedefs
      •  t_fp_OodleNet_Plugin_MallocAligned
      •  t_fp_OodleNet_Plugin_Free
      •  t_fp_OodleNet_Plugin_RunJob
      •  t_fp_OodleNet_Plugin_WaitJob
      •  t_fp_OodleNet_Plugin_Printf
      •  t_fp_OodleNet_Plugin_DisplayAssertion

 
  Oodle Network compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Rename_Async
Navigation
 OodleXIOQ_Delete_Async
 OodleXIOQ_MakeDir_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_Rename_Async( const char * fm,
                                     const char * to,
                                     OO_BOOL overwrite,
                                     OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                     OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                     const OodleXHandle * dependencies OODEFAULT( NULL ),
                                     OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a renamefile request.
Parameters
fm  the file to rename (VFS, UTF-8)
to  the new file name (VFS, UTF-8)
overwrite  if true, any existing file of name "to" will be overwrriten
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Asynchronously rename a file.

Use OodleXIOQ_ForceWriteable_Async (on the to name) before the rename to force the ovewriting of read-only and other no-access conditions.
 

OodleXIOQ_Delete_AsyncOodleX low level async ioOodleXIOQ_MakeDir_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on PS4
Navigation
 About Oodle on Windows
 About Oodle on Platforms
 About Oodle on Nintendo Switch
 Welcome to Oodle
 Change Log

Oodle for PS4 is provided as a lib.

lib/liboo2coreps4.a
lib/liboo2extps4.a

The debug build of the Oodle lib is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems.


OodleX file IO on PS4 goes through the sceKernel low level routines, not FIOS.

OodleX Malloc on PS4 uses sceKernelAllocateDirectMemory for large allocations, and clib malloc for small ones.

NOTE : to build the examples on PS4 without worrying about running out of memory, I use :

ifdef _RADPS4_ unsigned int sceLibcHeapExtendedAlloc = 1; /* Switch to dynamic allocation */ size_t sceLibcHeapSize = SCE_LIBC_HEAP_SIZE_EXTENDED_ALLOC_NO_LIMIT; /* no upper limit for heap area */

endif


Do not decompress directly into uncached graphics memory. See FAQ: How do I decompress to graphics memory quickly?.

Oodle now automatically detects this and warns about it.
 

About Oodle on Windows UWPAbout Oodle on PlatformsAbout Oodle on Nintendo Switch

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX threading util
Navigation
 OodleX async handle operations
 Oodle2 Ext API Documentation
 OodleX low level async io
 Welcome to Oodle
 Change Log

Threading utilities.

  •  Functions
    •  OodleX_Semaphore_Post
    •  OodleX_Semaphore_Wait
    •  OodleX_CreateThread
    •  OodleX_WaitAndDestroyThread
    •  OodleX_ReleaseThreadTLS
    •  OodleX_CorePlugin_RunJob
    •  OodleX_CorePlugin_WaitJob
    •  OodleX_GetNumWorkerThreads
  •  Typedefs
    •  OodleX_Semaphore
    •  OodleX_ThreadFunc

 
OodleXHandleCountdown_DecrementOodle2 Ext API DocumentationOodleX_Semaphore_Post

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleCore_Plugins_SetPrintf
Navigation
 OodleCore_Plugins_SetJobSystemAndCount
 OodleCore_Plugins_SetAssertion
 Welcome to Oodle
 Change Log
// Function prototype:
t_fp_OodleCore_Plugin_Printf * OodleCore_Plugins_SetPrintf( t_fp_OodleCore_Plugin_Printf * fp_rrRawPrintf );
Discussion
Install the callback used by Oodle Core for logging
Parameters
fp_rrRawPrintf  function pointer to your log function; may be NULL to disable all logging
Return Value
return  returns the previous function pointer
Discussion

Use this function to install your own printf for Oodle Core.

The default implementation in debug builds, if you install nothing, uses the C stdio printf for logging. On Microsoft platforms, it uses OutputDebugString and not stdio.

To disable all logging, call OodleCore_Plugins_SetPrintf(NULL)

WARNING : this function is NOT thread safe! It should be done only once and done in a place where the caller can guarantee thread safety.

In the debug build of Oodle, you can install OodleCore_Plugin_Printf_Verbose to get more verbose logging


 

OodleCore_Plugins_SetJobSystemAndCountCore pluginsOodleCore_Plugins_SetAssertion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Rename_AsyncAndWait
Navigation
 OodleXIOQ_Delete_AsyncAndWait
 OodleXIOQ_GetFileSize_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_Rename_AsyncAndWait( const char * fm,
                                       const char * to,
                                       OO_BOOL overwrite );
Discussion
See OodleXIOQ_Rename_Async
 
OodleXIOQ_Delete_AsyncAndWaitOodleX low level async ioOodleXIOQ_GetFileSize_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_IOAligned
Navigation
 OodleXMalloc_ValidatePointer
 OodleXFree_IOAligned
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXMalloc_IOAligned( OO_SINTa size );
Discussion
OodleXMalloc_IOAligned result is guaranteed to be aligned to OODLEX_IO_MAX_ALIGNMENT
Parameters
size  bytes to allocate ; will be aligned up to OODLEX_IO_MAX_ALIGNMENT !
Return Value
return  pointer to the allocated memory
Discussion

OodleXMalloc_IOAligned should be used to get memory that can be used in OodleIOQ and other places that require disk-aligned pointers. OodleXMalloc_IOAligned may just pass through to OodleXMallocBig provided by the client, or it may not if the OodleXMallocBigAlignment is very large.
 

OodleXMalloc_ValidatePointerOodleX Memory AllocatorsOodleXFree_IOAligned

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetAvailableAsyncSelect
Navigation
 OodleX_SetHandleAutoDelete
 OodleXHandleEvent_Alloc
 Welcome to Oodle
 Change Log
// Function prototype:
OO_U32 OodleX_GetAvailableAsyncSelect( );
Discussion
Get the currently available async systems
Return Value
return  a bitwise OR of OodleXAsyncSelect flags
Discussion

The OodleXAsyncSelect_Wide bit is set if there is more than one runner available.
 

OodleX_SetHandleAutoDeleteOodleX async handle operationsOodleXHandleEvent_Alloc

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetLastPendingOpOnFile
Navigation
 OodleXIOQ_Wait_GetInfo
 OodleXIOQ_GetName
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_GetLastPendingOpOnFile( OodleXIOQFile file );
Discussion
Get an operation on this file, if any
Parameters
file  the IOQFile to query
Return Value
return  the operation found, or 0 if none
Discussion

The operation returned may no longer be pending (nor the last) by the time you check it.
 

OodleXIOQ_Wait_GetInfoOodleX low level async ioOodleXIOQ_GetName

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_S64_to_SINTa_check
Navigation
 OodleX_IOAlignDownSINTa
 OodleX_GetSeconds
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleX_S64_to_SINTa_check( const OO_S64 x );
Discussion
Convert OO_S64 to OO_SINTa and check
Discussion
Used for loading 64-bit file sizes into memory buffers. Converts type and checks that file size fits in memory.

In 64 bit builds, this is a no-op. On 32 bit builds it ensures you to don't lose bits in the cast accidentally.

See also FAQ: What is SINTa? How do I load files bigger than 2 GB?.
 

OodleX_IOAlignDownSINTaOodleX UtilsOodleX_GetSeconds

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_RunJob
Navigation
 t_fp_OodleNet_Plugin_Free
 t_fp_OodleNet_Plugin_WaitJob
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC OO_U64( OODLE_CALLBACK t_fp_OodleNet_Plugin_RunJob )( t_fp_Oodle_Job * fp_job,
                  void * job_data,
                  OO_U64 * dependencies,
                  int num_dependencies,
                  void * user_ptr );
Discussion
Function pointer type for OodleNet_Plugins_SetJobSystem
Parameters
dependencies  array of handles of other pending jobs. All guaranteed to be nonzero.
num_dependencies  number of dependencies. Guaranteed to be no more than OODLE_JOB_MAX_DEPENDENCIES.
user_ptr  is passed through from the OodleLZ_CompressOptions.
Return Value
return  handle to the async job, or 0 if it was run synchronously
Discussion

RunJob will call fp_job(job_data)

it may be done on a thread, or it may run the function synchronously and return 0, indicating the job is already done. The returned OO_U64 is a handle passed to WaitJob, unless it is 0, in which case WaitJob won't get called.

fp_job should not run until all the dependencies are done. This function should not delete the dependencies.

RunJob must be callable from within an Oodle Job, i.e. jobs may spawn their own sub-jobs directly. However, the matching WaitJob calls will only ever occur on the thread that called the internally threaded Oodle API function.

See About Oodle Job Threading Plugins
 

t_fp_OodleNet_Plugin_FreeNetwork pluginst_fp_OodleNet_Plugin_WaitJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How do I decompress to graphics memory quickly?
Navigation
 FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
 Frequently Asked Questions
 FAQ: How do I get the Oodle logs?
 Welcome to Oodle
 Change Log

The OodleLZ decompressors must read back from the buffer that they are decompressing into.

If you try to decompress directly into uncached memory (or write combined memory, such as graphics buffers), you may see very bad performance. Several of Oodle's supported targets don't offer an efficient way to check memory types underlying virtual addresses; instead, Oodle performs a quick check to determine how fast repeated byte reads from the same location (the first byte of the destination buffer) are and logs a warning when they are repeatedly very slow. Generally, this means the destination is uncached.

There are a few ways to decode for graphics, depending on your platform and exact usage.

The simplest way is to decompress into a temporary buffer of normal cached memory, and then memcpy from there into the graphics memory.

For unbounded-window LZ, you would decode into a cached buffer equal to the size of the target. You can copy this buffer to the final graphics memory either all at once after decoding, or incrementally after each LZ BLOCK or QUANTUM decode from DecodeSome.

You can also use a bounded window by setting the dictionarySize CompressOption, in which case you can use a sliding window of cached CPU memory, and copy out chunks to the final target graphics memory as you decode.

See lz_test_11 in example_lz : Example demonstrating LZ compression and decompression for a demonstration of this style of decompression.

On platforms where it is possible, it can be fastest to set your buffer memory type to cached, do the Oodle decompression into that buffer, then flush the cache, then change the memory type to uncached/GPU. You must ensure the buffer is not sent to the GPU until the cache flush is done.

When loading levels, ideally you'd do lots of LZ decodes and then only flush the cache once, not after every resource is decoded.

On the PS4 you could decode into WB_ONION memory, and after the decode is complete, change the memory to WB_GARLIC using sceKernelMtypeprotect or sceKernelBatchMap.

On the Xbox One you can reserve a range of memory with VirtualAlloc, commit it as READWRITE and cached on the CPU to do the LZ decode, then decommit the memory and commit the same address again as GPU_READONLY. This is a workaround which allows you to change the cache type of a memory range (because VirtualProtect does not allow it).

Contact Oodle support for help if you wish to explore these advanced memory type manipulations.

NOTE : on some platforms, Oodle now automatically detects it if you try to decode into uncached memory, and logs a usage warning about it. See Oodle_SetUsageWarnings
 

FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?Frequently Asked QuestionsFAQ: How do I get the Oodle logs?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZSeekTable_Flags
Navigation
 OodleLZ_FuzzSafe
 OodleLZ_CompressScratchMemBoundType
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZSeekTable_Flags
{
    OodleLZSeekTable_Flags_None = 0,
    OodleLZSeekTable_Flags_MakeRawCRCs = 1,
    OodleLZSeekTable_Flags_Force32 = 0x40000000
};
Discussion
Options for OodleLZ_CreateSeekTable
Enumerants
OodleLZSeekTable_Flags_None  default
OodleLZSeekTable_Flags_MakeRawCRCs  make the rawCRCs member of OodleLZ_SeekTable
OodleLZSeekTable_Flags_Force32 

 
OodleLZ_FuzzSafeOodleAPI_LZ_CompressorsOodleLZ_CompressScratchMemBoundType

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLENETWORK1_HASH_BITS_DEFAULT
Navigation
 OODLENETWORK1_MAX_DICTIONARY_SIZE
 OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLENETWORK1_HASH_BITS_DEFAULT (19)
Discussion
Good default value for OodleNetwork1 hash table size log2
 
OODLENETWORK1_MAX_DICTIONARY_SIZEOodleAPI_OodleNetwork1OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_ASYNC_HANDLE_DONE
Navigation
 OODLEX_ASYNC_HANDLE_PENDING
 OODLEX_ASYNC_HANDLE_ERROR
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_ASYNC_HANDLE_DONE ((OodleXHandle)0x0000000100000001ULL)
Discussion
OodleXHandle to a special always-done handle. Calls to OodleX_GetStatus on this handle value will return &OodleXStatus_Done. This handle must not be deleted! Do not call OodleX_Wait on it with deleteIfDone = true.
 
OODLEX_ASYNC_HANDLE_PENDINGOodleX async handle operationsOODLEX_ASYNC_HANDLE_ERROR

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_MakeDir_Async
Navigation
 OodleXIOQ_Rename_Async
 OodleXIOQ_FreeBufferIOAligned_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_MakeDir_Async( const char * name,
                                      OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                      OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                      const OodleXHandle * dependencies OODEFAULT( NULL ),
                                      OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a make dir request.
Parameters
name  the dir to make (VFS, UTF-8)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Asynchronously make a dir.
 

OodleXIOQ_Rename_AsyncOodleX low level async ioOodleXIOQ_FreeBufferIOAligned_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Core plugins
Navigation
 Core Base
 Oodle2 Core API Documentation
 Core LZ compression
 Welcome to Oodle
 Change Log
  •  Functions
    •  OodleCore_Plugins_SetAllocators
    •  OodleCore_Plugins_SetJobSystem
    •  OodleCore_Plugins_SetJobSystemAndCount
    •  OodleCore_Plugins_SetPrintf
    •  OodleCore_Plugins_SetAssertion
  •  Typedefs
    •  t_fp_OodleCore_Plugin_MallocAligned
    •  t_fp_OodleCore_Plugin_Free
    •  t_fp_OodleCore_Plugin_RunJob
    •  t_fp_OodleCore_Plugin_WaitJob
    •  t_fp_OodleCore_Plugin_Printf
    •  t_fp_OodleCore_Plugin_DisplayAssertion

 
Oodle_LogHeaderOodle2 Core API DocumentationOodleCore_Plugins_SetAllocators

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_FreeBufferIOAligned_Async
Navigation
 OodleXIOQ_MakeDir_Async
 OodleXIOQ_GetInfoByName_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_FreeBufferIOAligned_Async( OodleXIOQFile file,
                                                  void * buffer,
                                                  OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                  OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                  const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                  OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a free-buffer request
Parameters
file  the request is scheduled on this file
buffer  the buffer to free
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Runs OodleXFree_IOAligned as an IO operation on the file's sequential list of ops.

This is mainly useful with OodleXIOQ_Write_Async. When you write a buffer, you can't free it until the write is done, with this you can just call Write and then FreeBuffer ; the FreeBuffer will be run when the Write is done.
 

OodleXIOQ_MakeDir_AsyncOodleX low level async ioOodleXIOQ_GetInfoByName_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_SetVerboseLevel
Navigation
 OodleXLog_GetVerboseLevel
 OodleXLog_Flush
 Welcome to Oodle
 Change Log
// Function prototype:
int OodleXLog_SetVerboseLevel( int v );
Discussion
Set the global verbose level
Parameters
v  verbose level to use
Return Value
return  the previous verbose level
Discussion

The verbose level should typically be one of OodleXLog_VerboseLevel , but that's not required.

If the verbose level is N, calls to OodleXLog_Printf(M,..) will be skipped any time M > N. eg. at the default development level of OodleXLog_Verbose_Some (1), all logs will be output except for OodleXLog_Printf_v2.


 

OodleXLog_GetVerboseLevelOodleX Debug aidsOodleXLog_Flush

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX low level async io
Navigation
 OodleX threading util
 Oodle2 Ext API Documentation
 OodleX Debug aids
 Welcome to Oodle
 Change Log

OodleIOQ low level, stateless, asychronous IO.

  •  About OodleIOQ
  •  Defines
    •  OODLEX_IO_MAX_ALIGNMENT
    •  OODLEX_BUFFER_SIZE_DEFAULT
    •  OODLEX_FILEINFO_FLAG_INVALID
    •  OODLEX_FILEINFO_MODTIME_INVALID
    •  OODLEX_FILE_SIZE_INVALID
    •  OODLEX_FILE_OPEN_NO_RESERVE_SIZE
    •  OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
  •  Enumerants
    •  OodleXCopyFileFlags
    •  OodleFileNotFoundIsAnError
    •  OODLEX_FILEINFO_FLAGS
    •  OodleXError
    •  OodleXFileMode
    •  OodleXFileOpenFlags
  •  Structures
    •  OodleXFileOpsVTable
    •  OodleXFileInfo
  •  Functions
    •  OodleXIOQ_WaitDoneAllPending
    •  OodleXIOQ_KickAnyDelayed
    •  OodleXIOQ_GetStatus
    •  OodleXIOQ_GetErrorDetails
    •  OodleXIOQ_GetErrorEnum
    •  OodleXIOQ_LogError
    •  OodleXIOQ_Wait
    •  OodleXIOQ_GetInfo
    •  OodleXIOQ_Wait_GetInfo
    •  OodleXIOQ_GetLastPendingOpOnFile
    •  OodleXIOQ_GetName
    •  OodleXIOQ_GetLastError
    •  OodleXIOQ_ClearError
    •  OodleXIOQ_LogLastError
    •  OodleXIOQ_GetOSHandle
    •  OodleXIOQ_SetVTable
    •  OodleXIOQ_Fence_Async
    •  OodleXIOQ_OpenForRead_Async
    •  OodleXIOQ_OpenAndRead_Async
    •  OodleXIOQ_OpenForWriteCreate_Async
    •  OodleXIOQ_OpenForWriteTempName_Async
    •  OodleXIOQ_CloseFile_Async
    •  OodleXIOQ_CloseFileRename_Async
    •  OodleXIOQ_Read_Async
    •  OodleXIOQ_Write_Async
    •  OodleXIOQ_SetFileSize_Async
    •  OodleXIOQ_ReserveFileSizeForWrite_Async
    •  OodleXIOQ_ForceWriteable_Async
    •  OodleXIOQ_Delete_Async
    •  OodleXIOQ_Rename_Async
    •  OodleXIOQ_MakeDir_Async
    •  OodleXIOQ_FreeBufferIOAligned_Async
    •  OodleXIOQ_GetInfoByName_Async
    •  OodleXIOQ_GetInfoByName_GetResult
    •  OodleXIOQ_SetInfoByName_Async
    •  OodleXIOQ_ReadMallocWholeFile_Async
    •  OodleXIOQ_ReadMallocWholeFile_GetResult
    •  OodleXIOQ_OpenAndReadMallocWholeFile_Async
    •  OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
    •  OodleXIOQ_OpenWriteWholeFileClose_Async
    •  OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
    •  OodleXIOQ_ReadUnalignedAdjustPointer_Async
    •  OodleXIOQ_MakeAllDirs_Async
    •  OodleXIOQ_CopyFile_Async
    •  OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
    •  OodleXIOQ_WriteWholeFile_AsyncAndWait
    •  OodleXIOQ_CopyFile_AsyncAndWait
    •  OodleXIOQ_GetInfoByName_AsyncAndWait
    •  OodleXIOQ_SetInfoByName_AsyncAndWait
    •  OodleXIOQ_MakeAllDirs_AsyncAndWait
    •  OodleXIOQ_Delete_AsyncAndWait
    •  OodleXIOQ_Rename_AsyncAndWait
    •  OodleXIOQ_GetFileSize_AsyncAndWait
    •  OodleXIOQ_NameIsDir_AsyncAndWait
    •  OodleX_GetOSFileOps
    •  OodleX_GetDefaultFileOps
    •  OodleX_SetDefaultFileOps
  •  Typedefs
    •  OodleXIOQFile
    •  OodleXOSFile
    •  OodleXOSFileListing

 
OodleX_GetNumWorkerThreadsOodle2 Ext API DocumentationAbout OodleIOQ

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleNetwork1
Navigation
 OodleAPI_OodleNetwork1
 Welcome to Oodle
 Change Log

OodleNetwork1 is a specialized compressor designed for compression of network packet transmission. It's designed to reduce game bandwidth use and improve player experience.


Quick Intro to OodleNetwork features

  • OodleNetwork encodes packet by packet with zero latency. The runtime encode and decode are simple buffer->buffer calls. Just build your packet as usual and then compress it, and send the compressed packet.

  • OodleNetwork runtime integration is extremely simple, just call Encode() in the server and Decode() in the client (for one-way compression). The complexity is all in the offline training process.

  • OodleNetwork does its heavy work in the offline training phase, so the runtime can be quite fast.

  • OodleNetwork does no allocations at runtime (you provide the memory for the model). OodleNetwork for UDP uses zero memory per channel. OodleNetwork for TCP uses around 100k per channel, which may be provided by the client code as a linear block.

  • OodleNetwork is about the same speed as "zlib -5" or "LZ4 HC" while achieving much more compression.

  • OodleNetwork is able to compress packets even if they have already been bit-packed.

  • OodleNetwork will never expand a packet.

  • OodleNetwork uses a dictionary size of your choosing. Even a small 1 MB dictionary can provide good compression. A typical size is 4 MB. Larger dictionaries generally give more compression, but you are free to choose the tradeoff that best suits your needs.

  • OodleNetwork is in Oodle2 Core and the runtime available on all platforms. (model training can only be done on desktop platforms)

  • OodleNetwork is fully thread-safe and re-entrant with no mutexes or blocking. The shared model data is read-only after initialization.


Compression of UDP network packets

With UDP networking, packets may be dropped or arrive in different orders. This means there is not a consistent history that the encoder and decoder see, so you cannot use any compressor which is based on a per-channel history (zlib, OodleLZ, OodleNetwork1-TCP).

OodleNetwork1UDP does compression of packets without any per-channel state.

The OodleNetwork1UDP_State is global and const; you make it offline in training, then distribute it as a const block of data with your game. It can be used by all encoder and decoder channels.

Sometimes it may be advantageous to have separate models for upstream or downstream traffic, or to compress only one direction and not the other. You can easily try these options and choose what works best for you.

Compression is done on each packet independently, so they can be lost or out of order and still decompress correctly.

OodleNetwork1 [dictionary MB|hash bits]

OodleNetwork1 for TCP with per-channel history :

OodleNetwork1 [8|19] : 595654217 -> 123101634 = 4.839:1
1605378 packets; 371.0 -> 76.7 average

OodleNetwork1 for UDP with no per-channel state :

OodleNetwork1UDP [8|19] : 595654217 -> 150022411 = 3.970:1
1605378 packets; 371.0 -> 93.4 average


Compression of TCP network packets

NOTE : OodleNetwork1 and OodleLZ are for TCP network compression, that is when you have a reliable per-channel history. For UDP, use OodleNetwork1UDP (see below).

OodleNetwork1 for TCP uses a shared static dictionary, plus a dynamic state per channel.

Usage of OodleNetwork1 is demonstrated in example_packet : Example demonstrating network packet compression.

For more normal data distribution needs, and large downloads or join packets, use Oodle Data Compression. See About OodleLZ.

OodleNetwork1 is "zero latency". That is, it don't add any buffering or delay of packets. It produces compressed bytes immediately for each raw byte processed.

OodleNetwork1 uses a static dictionary and hash table which is const and shared by all network channels. The size is set by the user. The bigger the static dictionary, the more compression you will get. There is an adaptive per-channel arithmetic coder so that the match length and literal statistics can adapt to the channel a bit (this was a big win vs. using any kind of static models).

OodleNetwork1 has only 104k of per-channel state. (compare to zlib which uses 400k per encoder)

On the server, a large static dictionary is no problem. They're running 16GB servers with 10,000 connections, they really don't care if the static dictionary is 64MB. However, that same static dictionary also has to be on the client, so the limit on how big a static dictionary you can use really comes from the client side. I suspect that something in the 8MB - 16MB range is reasonable. (and of course you can compress the static dictionary; it's only something like 2-4 MB that you have to distribute and load). (for loading the compressed static dictionary off disk, see About OodleLZ)

(BTW you don't necessarily need an adaptive compression state for every open channel. If some channels tend to go idle, you could drop their state. When the channel starts up again, grab a fresh state (and send a reset message to the client so it wipes its adaptive state). You could do something like have a few thousand compression states which you cycle in an LRU for an unbounded number of open channels. Of course the problem with that is if you actually get a higher number of simultaneous active connections you would be recycling states all the time, which is just the standard cache over-commit problem that causes nasty thrashing, so YMMV etc.)

Here are some real world results :

OodleNetwork1 TCP [dictionary MB|hash bits]

OodleNetwork1 [4|18] : 595654217 -> 131935361 = 4.515:1
1605378 packets; 371.0 -> 82.2 average
OodleNetwork1 [8|19] : 595654217 -> 123101634 = 4.839:1
1605378 packets; 371.0 -> 76.7 average
OodleNetwork1 [16|20] : 595654217 -> 110427772 = 5.394:1
1605378 packets; 371.0 -> 68.8 average
OodleNetwork1 [32|21] : 595654217 -> 93276137 = 6.386:1
1605378 packets; 371.0 -> 58.1 average


See OodleAPI_OodleNetwork1 , also Capturing Training data for OodleNetwork and Forming Packets for Maximum Compression
 

OodleAPI_OodleNetwork1OodleAPI_OodleNetwork1OODLENETWORK1_MAX_DICTIONARY_SIZE

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_GetDefaults_DebugSystems
Navigation
 OodleX_Init_GetDefaults_Threads
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleX_Init_GetDefaults_DebugSystems
{
    OodleX_Init_GetDefaults_DebugSystems_No = 0,
    OodleX_Init_GetDefaults_DebugSystems_Yes = 1,
    OodleX_Init_GetDefaults_DebugSystems_Force32 = 0x40000000
};
Discussion
Should GetDefaults enable debugging systems?
 
OODLE_WORKERS_COUNT_ALL_HYPER_CORESOodleX Startup and ShutdownOodleX_Init_GetDefaults_Threads

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor
Navigation
 OodleLZ_Verbosity
 OodleLZ_PackedRawOverlap
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_Compressor
{
    OodleLZ_Compressor_Invalid = -1,
    OodleLZ_Compressor_None = 3,
    OodleLZ_Compressor_Kraken = 8,
    OodleLZ_Compressor_Leviathan = 13,
    OodleLZ_Compressor_Mermaid = 9,
    OodleLZ_Compressor_Selkie = 11,
    OodleLZ_Compressor_Hydra = 12,
    OodleLZ_Compressor_BitKnit = 10,
    OodleLZ_Compressor_LZB16 = 4,
    OodleLZ_Compressor_LZNA = 7,
    OodleLZ_Compressor_LZH = 0,
    OodleLZ_Compressor_LZHLW = 1,
    OodleLZ_Compressor_LZNIB = 2,
    OodleLZ_Compressor_LZBLW = 5,
    OodleLZ_Compressor_LZA = 6,
    OodleLZ_Compressor_Count = 14,
    OodleLZ_Compressor_Force32 = 0x40000000
};
Discussion
Selection of compression algorithm.
Enumerants
OodleLZ_Compressor_Invalid 
OodleLZ_Compressor_None  None = memcpy, pass through uncompressed bytes
OodleLZ_Compressor_Kraken  Fast decompression and high compression ratios, amazing!
OodleLZ_Compressor_Leviathan  Leviathan = Kraken's big brother with higher compression, slightly slower decompression.
OodleLZ_Compressor_Mermaid  Mermaid is between Kraken & Selkie - crazy fast, still decent compression.
OodleLZ_Compressor_Selkie  Selkie is a super-fast relative of Mermaid. For maximum decode speed.
OodleLZ_Compressor_Hydra  Hydra, the many-headed beast = Leviathan, Kraken, Mermaid, or Selkie (see About OodleLZ Hydra)
OodleLZ_Compressor_BitKnit  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZB16  DEPRECATED but still supported
OodleLZ_Compressor_LZNA  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZH  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZHLW  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZNIB  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZBLW  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_LZA  no longer supported as of Oodle 2.9.0
OodleLZ_Compressor_Count 
OodleLZ_Compressor_Force32 
Discussion
Each compressor provides a different balance of speed vs compression ratio.

New Oodle users should only use the new sea monster family of compressors.

The OODLE_ALLOW_DEPRECATED_COMPRESSORS set of compressors is no longer supported as of Oodle 2.9.0 ; see FAQ: What are the Oodle deprecated compressors ?

The sea monsters are all fuzz safe and use whole-block quantum (not the 16k quantum) (OodleLZ_Compressor_UsesWholeBlockQuantum)

If you need to encode the deprecated compressors, define OODLE_ALLOW_DEPRECATED_COMPRESSORS before including oodle2.h

See FAQ: Which OodleLZ should I use? for a quick FAQ on which compressor to use

See About OodleLZ for discussion of how to choose a compressor.
 

OodleLZ_VerbosityOodleAPI_LZ_CompressorsOodleLZ_PackedRawOverlap

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressOptions
Navigation
 OodleLZ_DecodeSome_Out
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleLZ_CompressOptions
{
    OO_U32              unused_was_verbosity;
    OO_S32              minMatchLen;
    OO_BOOL             seekChunkReset;
    OO_S32              seekChunkLen;
    OodleLZ_Profile     profile;
    OO_S32              dictionarySize;
    OO_S32              spaceSpeedTradeoffBytes;
    OO_S32              unused_was_maxHuffmansPerChunk;
    OO_BOOL             sendQuantumCRCs;
    OO_S32              maxLocalDictionarySize;
    OO_BOOL             makeLongRangeMatcher;
    OO_S32              matchTableSizeLog2;
    OodleLZ_Jobify      jobify;
    void *              jobifyUserPtr;
    OO_S32              farMatchMinLen;
    OO_S32              farMatchOffsetLog2;
    OO_U32              reserved[4];
};
Discussion
Options for the compressor
Members
unused_was_verbosity  unused ; was verbosity (set to zero)
minMatchLen  minimum match length ; cannot be used to reduce a compressor's default MML, but can be higher. On some types of data, a large MML (6 or 8) is a space-speed win.
seekChunkReset  whether chunks should be independent, for seeking and parallelism
seekChunkLen  length of independent seek chunks (if seekChunkReset) ; must be a power of 2 and >= OODLELZ_BLOCK_LEN ; you can use OodleLZ_MakeSeekChunkLen
profile  decoder profile to target (set to zero)
dictionarySize  sets a maximum offset for matches, if lower than the maximum the format supports. <= 0 means infinite (use whole buffer). Often power of 2 but doesn't have to be.
spaceSpeedTradeoffBytes  this is a number of bytes; I must gain at least this many bytes of compressed size to accept a speed-decreasing decision
unused_was_maxHuffmansPerChunk  unused ; was maxHuffmansPerChunk
sendQuantumCRCs  should the encoder send a CRC of each compressed quantum, for integrity checks; this is necessary if you want to use OodleLZ_CheckCRC_Yes on decode
maxLocalDictionarySize  (Optimals) size of local dictionary before needing a long range matcher. This does not set a window size for the decoder; it's useful to limit memory use and time taken in the encoder. maxLocalDictionarySize must be a power of 2. Must be <= OODLELZ_LOCALDICTIONARYSIZE_MAX
makeLongRangeMatcher  (Optimals) should the encoder find matches beyond maxLocalDictionarySize using an LRM
matchTableSizeLog2 (non-Optimals) when variable, sets the size of the match finder structure (often a hash table) ; use 0 for the compressor's default
jobify  controls internal job usage by compressors
jobifyUserPtr  user pointer passed through to RunJob and WaitJob callbacks
farMatchMinLen  far matches must be at least this len
farMatchOffsetLog2  if not zero, the log2 of an offset that must meet farMatchMinLen
reserved  reserved space for adding more options; zero these!
Discussion
Typically filled by calling OodleLZ_CompressOptions_GetDefault , then individual options may be modified, like :

OodleLZ_CompressOptions my_options = OodleLZ_CompressOptions_GetDefault()

To ensure you have set up the options correctly, call OodleLZ_CompressOptions_Validate.

unused_was_verbosity : place holder, set to zero

minMatchLen : rarely useful. Default value of 0 means let the compressor decide. On some types of data, bumping this up to 4,6, or 8 can improve decode speed with little effect on compression ratio. Most of the Oodle compressors use a default MML of 4 at levels below 7, and MML 3 at levels >= 7. If you want to keep MML 4 at the higher levels, set minMatchLen here to 4. minMatchLen cannot be used to reduce the base MML of the compressor, only to increase it.

seekChunkReset must be true if you want the decode to be able to run "Wide", with pieces that can be decoded independently (not keeping previous pieces in memory for match references).

seekChunkLen : length of independent seek chunks (if seekChunkReset) ; must be a power of 2 and >= OODLELZ_BLOCK_LEN ; you can use OodleLZ_MakeSeekChunkLen

profile : tells the encoder to target alternate bitstream profile. Default value of zero for normal use.

dictionarySize : limits the encoder to partial buffer access for matches. Can be useful for decoding incrementally without keeping the entire output buffer in memory.

spaceSpeedTradeoffBytes is a way to trade off compression ratio for decode speed. If you make it smaller, you get more compression ratio and slower decodes. It's the number of bytes that a decision must save to be worth a slower decode. Default is 256 (OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT). So that means the encoder must be able to save >= 256 bytes to accept something that will slow down decoding (like adding another Huffman table). The typical range is 64-1024.

Lower spaceSpeedTradeoffBytes = more compression, slower decode Higher spaceSpeedTradeoffBytes = less compression, faster decode

spaceSpeedTradeoffBytes is the primary parameter for controlling Hydra. The default value of 256 will make Hydra decodes that are just a little bit faster than Kraken. You get Kraken speeds around 200, and Mermaid speeds around 1200.

At the extreme, a spaceSpeedTradeoffBytes of zero would mean all you care about is compression ratio, not decode speed, you want the encoder to make the smallest possible output. (you cannot actually set zero, as zero values always mean "use default" in this struct; you never really want zero anyway) Generally spaceSpeedTradeoffBytes below 16 provides diminishing gains in size with pointless decode speed loss.

spaceSpeedTradeoffBytes is on sort of powers of 2 scale, so you might want to experiment with 32,64,128,256,512

spaceSpeedTradeoffBytes outside the range [16 - 2048] is not recommended.

unused_was_maxHuffmansPerChunk : place holder, set to zero

sendQuantumCRCs : send hashes of the compressed data to verify in the decoder; not recommended, if you need data verification, use your own system outside of Oodle. DEPRECATED, not recommended. For backwards compatibility only.

maxLocalDictionarySize : only applies to optimal parsers at level >= Optimal2. This limits the encoder memory use. Making it larger = more compression, higher memory use. Matches within maxLocalDictionarySize are found exactly, outside the maxLocalDictionarySize window an approximate long range matcher is used.

makeLongRangeMatcher : whether an LRM should be used to find matches outside the maxLocalDictionarySize window (Optimal levels only)

matchTableSizeLog2 : for non-optimal levels (level <= Normal), controls the hash table size. Making this very small can sometimes boost encoder speed. For the very fastest encoding, use the SuperFast level and change matchTableSizeLog2 to 12 or 13.

matchTableSizeLog2 should usually be left zero to use the encoder's default

matchTableSizeLog2 allows you to limit memory use of the non-Optimal encoder levels. Memory use is roughly ( 1 MB + 4 << matchTableSizeLog2 )

jobify tells compressors how to use internal jobs for compression tasks. Jobs can be run in parallel using the job system plugins set with OodleCore_Plugins_SetJobSystem. Not all compressors or compression level support jobs, but the slower ones generally do. The default value of jobify is to use a thread system if one is installed.

farMatchMinLen and farMatchOffsetLog2 can be used to tune the encoded stream for a known cache size on the decoding hardware. If set, then offsets with log2 greater or each to farMatchOffsetLog2 must have a minimum length of farMatchMinLen. For example to target a machine with a 2 MB cache, set farMatchOffsetLog2 to 21, and farMatchMinLen to something large, like 16 or 20.

Without farMatchMinLen and farMatchOffsetLog2 set, the Oodle encoders tune for a blend of cache sizes that works well on most machines. dictionarySize can also be used to tune for cache size, but cuts off all matches beyond a certain distance. That may be more appropriate when you don't want to go out of cache at all. farMatchMinLen can only be used to make the standard blend target more restrictive; it can reduce the target cache size but can't make it larger (or it can raise min match len outside cache but can't make it shorter).

For help on setting up OodleLZ_CompressOptions contact support at oodle@radgametools.com

NOTE : fields you do not set should always be zero initialized. In particular the reserved fields should be zeroed. Zero always means "use default" and is a future-portable initialization value.

If you set fields to zero to mean "use default" you can call OodleLZ_CompressOptions_Validate to change them to default values. This is done automatically internally if you don't do it explicitly.


 

OodleLZ_CompressScratchMemBoundTypeOodleAPI_LZ_CompressorsOodleLZ_DecodeSome_Out

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Delete_AsyncAndWait
Navigation
 OodleXIOQ_MakeAllDirs_AsyncAndWait
 OodleXIOQ_Rename_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_Delete_AsyncAndWait( const char * path );
Discussion
See OodleXIOQ_Delete_Async
 
OodleXIOQ_MakeAllDirs_AsyncAndWaitOodleX low level async ioOodleXIOQ_Rename_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetStatus
Navigation
 OodleXIOQ_KickAnyDelayed
 OodleXIOQ_GetErrorDetails
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXIOQ_GetStatus( OodleXHandle req,
                                  OodleXHandleDeleteIfDone andDeleteIfDone,
                                  OO_U32 * pErrorCode OODEFAULT( NULL ),
                                  OO_S32 * pReturnValue OODEFAULT( NULL ) );
Discussion
Get the Status of a request, and optionally delete if done
Parameters
req  the IOQ operation handle to work on
andDeleteIfDone  if true and the returned status is >= Done the handle will be deleted
pErrorCode  (optional) the OS error code, if any
pReturnValue  (optional) the operation return value
Return Value
return  the status of the request
Discussion

This function is similar to OodleX_GetStatus, but for IOQ operation handles only, and it provides more information (optionally).

The error code returned can be processed with OodleXIOQ_GetErrorEnum or OodleXIOQ_GetErrorDetails.

The return value depends on the operation type. For example if the operation is a Read, it returns the number of bytes successfully read.
 

OodleXIOQ_KickAnyDelayedOodleX low level async ioOodleXIOQ_GetErrorDetails

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_ASYNC_HANDLE_INVALID
Navigation
 OODLEX_ASYNC_HANDLE_PENDING
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_ASYNC_HANDLE_INVALID ((OodleXHandle)0)
Discussion
OodleXHandle for an invalid handle. Calls to OodleX_GetStatus on this handle value will return &OodleXStatus_Invalid.
 
About OodleXHandleOodleX async handle operationsOODLEX_ASYNC_HANDLE_PENDING

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on IOS
Navigation
 About Oodle on Linux
 About Oodle on Platforms
 About Oodle on Android
 Welcome to Oodle
 Change Log

Oodle on IOS is currently provided as only the Oodle Core lib (no OodleX).

This includes the synchronous LZ compressors, as well as Oodle Network compression.

See Oodle2 Core vs Oodle2 Ext


Oodle on mobile does not include the Optimal level encoders. When OodleLZ_CompressionLevel_Optimal1 or higher is request, OodleLZ_CompressionLevel_Normal is used instead.
 

About Oodle on LinuxAbout Oodle on PlatformsAbout Oodle on Android

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_GetCallback
Navigation
 OodleXLog_SetCallback
 OodleXLog_GetVerboseLevel
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXLogCallbackRet * OodleXLog_GetCallback( );
Discussion
Get the OodleXLogCallbackRet currently set
Return Value
return  the global callback currently set
Discussion

See OodleXLog_SetCallback
 

OodleXLog_SetCallbackOodleX Debug aidsOodleXLog_GetVerboseLevel

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_packet : Example demonstrating network packet compression
Navigation
 example_network_client : Example with simple network client support
 Examples
 Welcome to Oodle
 Change Log
Discussion
Oodle example_packet

Example demonstrating network packet compression.

(See also example_network_client : Example with simple network client support for a simpler client-only example that's more suitable as a basis for your runtime)

The primary API is OodleNetwork1 (see About OodleNetwork1 and OodleAPI_OodleNetwork1).

For an overview see About Oodle Network Compression

To use example_packet, you must capture a sample of your game's network packets.

See Capturing Training data for OodleNetwork

This example can be used with Oodle Network lib only. If you wish to also apply LZ compression to the save model runtime data, toggle USE_OODLE_LZ_DATA_COMPRESSION.


#include "../include/oodle2net.h"

// optional, use Oodle LZ for the network model data
//#define USE_OODLE_LZ_DATA_COMPRESSION

#if defined(USE_OODLE_LZ_DATA_COMPRESSION) && ! defined(__OODLE2_H_INCLUDED__)
#include "../include/oodle2x.h"
#endif

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif

#if 1
// if the STL build gives you trouble, then set this to 0
// this is just used for extra diagnostic printing
#define EXAMPLE_PACKET_DO_SORT  1
#else
#define EXAMPLE_PACKET_DO_SORT  0
#endif

#define _ITERATOR_DEBUG_LEVEL 0
#define _HAS_ITERATOR_DEBUGGING 0

#if EXAMPLE_PACKET_DO_SORT
// try to get std::sort
// this is used for information display only
#include <algorithm>
#endif

#include "ooex.h" // example helpers

#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_packet
#endif

#ifdef _MSC_VER
#pragma warning(disable: 4505)
#endif


#ifndef __OODLE2_H_INCLUDED__
// if USE_OODLE_LZ_DATA_COMPRESSION off,
//  make a fake define of OodleLZ_Compressor_Invalid
//  so I can share the code path but toggle compression off
typedef enum OodleLZ_Compressor
{
    OodleLZ_Compressor_Invalid = -1,
    OodleLZ_Compressor_Force32 = 0x40000000
} OodleLZ_Compressor;
#endif

//===========================================================

This is where you #define options to select which compressors the example should test

To test OodleNetwork1

use EXAMPLE_PACKET_TEST_OODLENETWORK1_TCP for TCP networking or EXAMPLE_PACKET_TEST_OODLENETWORK1_UDP for UDP networking

If you set EXAMPLE_PACKET_TEST_OODLENETWORK1_DIC_SIZES, a variety of OodleNetwork1 dictionary sizes are tested.


// test OodleNetwork for UDP or TCP :
#define EXAMPLE_PACKET_TEST_OODLENETWORK1_TCP           0
#define EXAMPLE_PACKET_TEST_OODLENETWORK1_UDP           1
// test multiple dictionary sizes and report results
#define EXAMPLE_PACKET_TEST_OODLENETWORK1_DIC_SIZES     0

// enable Load Existing to repeat a test without re-training
#define EXAMPLE_PACKET_OODLENETWORK_LOAD_EXISTING   0
// Trials is slow but gives more compression
#define EXAMPLE_PACKET_OODLENETWORK_DO_TRIALS       0

//===========================================================
// get a vector :
#include <stddef.h>
#ifdef __GNUC__
#include <stdint.h>
#endif
#include <new> // for operator new placement new
#define CB_ALLOC(size)          malloc(size)
#define CB_FREE(ptr,size)       free(ptr)
#define CB_ASSERT_MALLOC(exp)   OOEX_ASSERT_ALWAYS(exp)
#include "cbvector.h"

//===========================================================
// little helper object to load a file and free it for us

#include "read_whole_file.h"

static OO_BOOL write_whole_file(const char * file_name,void * ptr,OO_SINTa size)
{
    FILE * fp = fopen(file_name,"wb");
    if ( ! fp ) return false;
    
    size_t count = fwrite(ptr,1,size,fp);
    
    fclose(fp);
    
    if ( count != (size_t)size ) return false;
    
    return true;
}

#ifndef BUILDING_TEST_PACKET

#define EXAMPLE_PACKET_DEFAULT_OUTPUT_PATH

namespace
{
struct OodleIOBuffer
{
    OO_U8 * ptr;
    OO_SINTa    size;
    
    explicit OodleIOBuffer(const char * name)
    {
        OodleXLog_Printf_v1("loading : %s ...\n",name); 

        ptr = (OO_U8 *) read_whole_file(name,&size);
        
        if ( ! ptr )
        {
            OodleXLog_Printf_v0("failed to read %s\n",name); 
            size = 0;
        }
    }
    
    
    explicit OodleIOBuffer(OO_SINTa alloc_size)
    {
        ptr = (OO_U8 *) malloc(alloc_size);
        size = alloc_size;
    }
    
    void Release()
    {
        if ( ptr )
        {
            free(ptr);
            ptr = NULL;
            size = 0;
        }
    }
    
    ~OodleIOBuffer()
    {
        Release();
    }
};
};
#endif

//===========================================================
// protos :
                
static void TestOodleNetwork1TCPPacketCoder( 
        const OodleIOBuffer & iob_packet_train ,
        const OodleIOBuffer & iob_packet_test ,
        int on1_dic_mb , int on1_ht_bits);
        
static void TestOodleNetwork1UDPPacketCoder( 
        const OodleIOBuffer & iob_packet_train ,
        const OodleIOBuffer & iob_packet_test ,
        int on1_dic_mb , int on1_ht_bits,
        int for_oodle_version_major);
                
//===========================================================
File names to find captured packets.

You should change these to the files you have captured.

You may also pass the file name and holdout fraction on the command line.

The holdout fraction should be adjusted to ensure you have enough packets for testing and training.



static const char * c_example_packet_file = "r:\\packet.bin"; // <- change this to your file name
static const int c_example_packet_test_holdout_fraction_denominator = 4; // hold out 1/4
// useful on very large samples :
//static const int c_example_packet_test_holdout_fraction_denominator = 2; // hold out 1/2

Some variables that control filtering of the packet stream, if you wish to reject some :

// to select a subset :
static const int c_packet_filter_skip_initial = 0;
static const int c_packet_filter_max_num = 1<<29;
//static const int c_packet_filter_skip_initial = 2500000;
//static const int c_packet_filter_max_num = 100000;

// to select a range of sizes :
//static const int c_packet_filter_min_size = 0;
//static const int c_packet_filter_max_size = 1<<29;
static const int c_packet_filter_min_size = 32; // exclude tiny packets
static const int c_packet_filter_max_size = 8192; // exclude large packets

example_packet_file should contain a capture of packets from a real game session. It should be at least 100 MB.

See Capturing Training data for OodleNetwork

Some portion (as set by example_packet_test_holdout_fraction_denominator) will be held out for testing the compression level. The remainder will be used for training. In real game usage, you don't need to hold out any for testing, this is just for evaluating the compression level. To be fair, the packets used in training are not used in testing.

The file format is :

packet.bin : OO_U32 [LE] : numbers of channels (num_channels) repeatedly : { OO_U32 [LE] : channel index in [0,num_channels-1] OO_U32 [LE] : number of bytes of data in this packet (num_bytes) OO_U8 * num_bytes : payload of this packet }

Channels are for TCP networking. For UDP, set num_channels = 1 and all channel indices to 0.


//===========================================================

static OO_U32 irandmod(OO_U32 size);

static void scan_packet_iobs(
        const OodleIOBuffer & iob_packet,
        OodleIOBuffer * p_iob_packet_test,
        OodleIOBuffer * p_iob_packet_train,
        int example_packet_test_holdout_fraction_denominator)
{
    const OO_U8 * packet_bin_buf = iob_packet.ptr;
    const OO_SINTa packet_bin_size = iob_packet.size;
    
    // get the number of channels ; first dword in packet bin : 
    const OO_U8 * packet_bin_ptr = packet_bin_buf;
    const OO_U8 * packet_bin_end = packet_bin_ptr + packet_bin_size;
    
    OodleIOBuffer & iob_packet_test = * p_iob_packet_test;
    
    OO_U8 * packet_test_ptr = iob_packet_test.ptr;
    OO_U8 * packet_train_ptr = p_iob_packet_train ? p_iob_packet_train->ptr : NULL;

    OO_S32 num_channels = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
    if ( num_channels == 0 ) num_channels = 1;
    OOEX_ASSERT( num_channels > 0 );
    
    OOEX_PUT32_LE(packet_test_ptr,num_channels); packet_test_ptr += sizeof(OO_U32);
    if ( packet_train_ptr )
    {
        OOEX_PUT32_LE(packet_train_ptr,num_channels); packet_train_ptr += sizeof(OO_U32);
    }

    OO_SINTa tot_packet_bytes = 0;
    OO_SINTa tot_num_packets = 0;
    OO_SINTa train_packet_bytes = 0;
    OO_SINTa train_num_packets = 0;
    OO_SINTa test_packet_bytes = 0;
    OO_SINTa test_num_packets = 0;
    OO_SINTa num_packets_excluded = 0;
    OO_SINTa packet_bytes_excluded = 0;
    OO_SINTa num_packets_initial = 0;

    OO_SINTa smallest_packet_size = 9999999;
    OO_SINTa largest_packet_size = 0;
    const OO_U8 * largest_packet_ptr = NULL;

    while( packet_bin_ptr < packet_bin_end )
    {
        // grab the current packet header :
        OO_S32 channel = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OO_S32 bytes = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_ASSERT( channel >= 0 && channel < num_channels );
        OOEX_ASSERT( bytes > 0 && bytes < (256*1024) );
    
        if ( num_packets_initial < c_packet_filter_skip_initial )
        {
            num_packets_initial++;
            if ( num_packets_initial == c_packet_filter_skip_initial )
            {
                OodleXLog_Printf_v1("skipped initial %d packets\n",c_packet_filter_skip_initial);
            }
            
            // skip it
            packet_bin_ptr += bytes;
            num_packets_excluded ++;
            packet_bytes_excluded += bytes;
            continue;
        }
    
        // get largest and smallest packet size *before* size filters :
        smallest_packet_size = OOEX_MIN(smallest_packet_size,bytes);
    
        if ( bytes > largest_packet_size )
        {
            largest_packet_size = bytes;
            largest_packet_ptr = packet_bin_ptr;
        }

        if ( bytes < c_packet_filter_min_size ||
            bytes > c_packet_filter_max_size )
        {
            // skip it
            packet_bin_ptr += bytes;
            num_packets_excluded ++;
            packet_bytes_excluded += bytes;
            continue;
        }               
    
        if ( packet_bin_ptr + bytes > packet_bin_end )
        {
            // partial end packet
            break;
        }
    
        tot_num_packets++;  
        tot_packet_bytes += bytes;
        
        if ( packet_train_ptr == NULL ||
            irandmod(example_packet_test_holdout_fraction_denominator) == 0 )
        {
            // test
        
            OOEX_PUT32_LE(packet_test_ptr,channel); packet_test_ptr += sizeof(OO_S32);
            OOEX_PUT32_LE(packet_test_ptr,bytes); packet_test_ptr += sizeof(OO_S32);
            
            memcpy(packet_test_ptr,packet_bin_ptr,bytes);
            
            packet_bin_ptr += bytes;
            packet_test_ptr += bytes;
            
            test_num_packets ++;
            test_packet_bytes += bytes;
        }
        else
        {
            //train
            
            OOEX_PUT32_LE(packet_train_ptr,channel); packet_train_ptr += sizeof(OO_S32);
            OOEX_PUT32_LE(packet_train_ptr,bytes); packet_train_ptr += sizeof(OO_S32);
            
            memcpy(packet_train_ptr,packet_bin_ptr,bytes);
            
            packet_bin_ptr += bytes;
            packet_train_ptr += bytes;
            
            train_num_packets ++;
            train_packet_bytes += bytes;
        }
        
        if ( tot_num_packets >= c_packet_filter_max_num )
            break;
    }

    iob_packet_test.size = packet_test_ptr - iob_packet_test.ptr;
    if ( p_iob_packet_train )
        p_iob_packet_train->size = packet_train_ptr - p_iob_packet_train->ptr;

    OodleXLog_Printf_v1("Read %d packets, %lld bytes (%.2f average)\n",(int)tot_num_packets,(long long)tot_packet_bytes,(double)tot_packet_bytes/tot_num_packets);
    OodleXLog_Printf_v1("Held out %d packets (%lld bytes) for testing, %d (%lld) for training\n",
        (int)test_num_packets,(long long)test_packet_bytes,
        (int)train_num_packets,(long long)train_packet_bytes);
        
    if ( num_packets_excluded > 0 )
    {
        OodleXLog_Printf_v1("%d packets excluded by filters (%d bytes)\n",(int)num_packets_excluded,
            (int)packet_bytes_excluded);
    }
    
    OodleXLog_Printf_v1("smallest_packet_size : %d (%d min filtered)\n",(int)smallest_packet_size,c_packet_filter_min_size);
    OodleXLog_Printf_v1("largest_packet_size : %d (%d max filtered)\n",(int)largest_packet_size,c_packet_filter_max_size);
    
    if ( largest_packet_size > c_packet_filter_max_size )
    {
        OodleXLog_Printf_v1("WARNING: large packets were found and excluded by filter.\n");
        OodleXLog_Printf_v1(" large packets SHOULD be compressed.  Consider using OodleLZ.\n");
    }

    #if 0
    // log the largest packet to see what it is
    if ( largest_packet_size > 0 )
    {
        OodleXIOQ_WriteWholeFile_AsyncAndWait("r:\\largest_packet.bin",largest_packet_ptr,largest_packet_size,OodleXFileOpenFlags_Buffered);
    }
    #else
    OOEX_UNUSED_VARIABLE(largest_packet_ptr);
    #endif
}
                     
static void test_packet_iobs(const OodleIOBuffer & iob_packet_test,
                     const OodleIOBuffer & iob_packet_train,
                     int for_oodle_version_major);
                     
static int example_packet_onefile(const char * example_packet_file,int example_packet_test_holdout_fraction_denominator,int for_oodle_version_major)
{
    OodleXLog_Printf_v1("Loading : \"%s\" , holding out 1/%d\n",example_packet_file,example_packet_test_holdout_fraction_denominator);

    if ( example_packet_test_holdout_fraction_denominator < 2 )
    {
        OodleXLog_Printf_v0("Error : invalid holdout fraction, must be >=2\n");
        return 10;
    }

    //=================================================
    
    {
        // load the files containing packet data :
        OodleIOBuffer iob_packet(example_packet_file);
        
        if ( ! iob_packet.ptr )
        {
            OodleXLog_Printf_v0("Failed to load : %s\n",example_packet_file);
            return 10;
        }
        
        //---------------------------------------
        // do test/train holdout :
        
        OodleIOBuffer iob_packet_test(iob_packet.size);
        OodleIOBuffer iob_packet_train(iob_packet.size);

        scan_packet_iobs(iob_packet,&iob_packet_test,&iob_packet_train,example_packet_test_holdout_fraction_denominator);
        iob_packet.Release();
        
        test_packet_iobs(iob_packet_test,iob_packet_train,for_oodle_version_major);

    }
    
    return 0;
}

#ifdef BUILDING_TEST_PACKET     
static int example_packet_twofile(const char * packet_train_file,const char * packet_test_file,int for_oodle_version_major)
{
    // load the files containing packet data :
    OodleIOBuffer iob_packet_test(packet_test_file);
        
    if ( ! iob_packet_test.ptr )
    {
        OodleXLog_Printf_v0("Failed to load : %s\n",packet_test_file);
        return 10;
    }

    OodleXLog_Printf_v1("Test  file (%s) : %d bytes\n",packet_test_file,(int)iob_packet_test.size);

    OodleIOBuffer iob_packet_test_parsed(iob_packet_test.size);
    
    // "scan" to apply exclusions and such :
    scan_packet_iobs(iob_packet_test,&iob_packet_test_parsed,NULL,0);
    iob_packet_test.Release();
    
    OodleIOBuffer iob_packet_train(packet_train_file);
    
    if ( ! iob_packet_train.ptr )
    {
        OodleXLog_Printf_v0("Failed to load : %s\n",packet_train_file);
        return 10;
    }   
    
    OodleXLog_Printf_v1("Train file (%s) : %d bytes\n",packet_train_file,(int)iob_packet_train.size);
        
    OodleIOBuffer iob_packet_train_parsed(iob_packet_train.size);
    scan_packet_iobs(iob_packet_train,&iob_packet_train_parsed,NULL,0);
    iob_packet_train.Release();
    
    test_packet_iobs(iob_packet_test_parsed,iob_packet_train_parsed,for_oodle_version_major);
    
    return 0;
}
#endif

static void test_packet_iobs(const OodleIOBuffer & iob_packet_test,
                     const OodleIOBuffer & iob_packet_train,
                     int for_oodle_version_major)
{
    //---------------------------
    
Run one or more tests based on the #defined options :
    
    //---------------------------
    if ( EXAMPLE_PACKET_TEST_OODLENETWORK1_TCP )
    {
        if ( EXAMPLE_PACKET_TEST_OODLENETWORK1_DIC_SIZES )
        {
            // test several typical on1 dictionary sizes
            TestOodleNetwork1TCPPacketCoder(iob_packet_train,iob_packet_test,1,17 );
            TestOodleNetwork1TCPPacketCoder(iob_packet_train,iob_packet_test,2,18 );
            TestOodleNetwork1TCPPacketCoder(iob_packet_train,iob_packet_test,4,19 );
            TestOodleNetwork1TCPPacketCoder(iob_packet_train,iob_packet_test,8,20 );
            TestOodleNetwork1TCPPacketCoder(iob_packet_train,iob_packet_test,16,21 );
        }
        else
        {
            int on1_dictionary_mb = 4;
            int on1_ht_bits = 18;
            //int on1_dictionary_mb = 8;
            //int on1_ht_bits = 19;
            //int on1_dictionary_mb = 16;
            //int on1_ht_bits = 20;
            
            TestOodleNetwork1TCPPacketCoder( 
                iob_packet_train,
                iob_packet_test,
                on1_dictionary_mb,
                on1_ht_bits );
        }
    }
    //---------------------------
    if ( EXAMPLE_PACKET_TEST_OODLENETWORK1_UDP )
    {
        if ( EXAMPLE_PACKET_TEST_OODLENETWORK1_DIC_SIZES )
        {
            //OodleXLog_SetVerboseLevel(0);
        
            TestOodleNetwork1UDPPacketCoder( iob_packet_train,iob_packet_test,1,17 ,for_oodle_version_major);                   
            TestOodleNetwork1UDPPacketCoder( iob_packet_train,iob_packet_test,2,18 ,for_oodle_version_major);                   
            TestOodleNetwork1UDPPacketCoder( iob_packet_train,iob_packet_test,4,19 ,for_oodle_version_major);                   
            TestOodleNetwork1UDPPacketCoder( iob_packet_train,iob_packet_test,8,20 ,for_oodle_version_major);                   
            TestOodleNetwork1UDPPacketCoder( iob_packet_train,iob_packet_test,16,21 ,for_oodle_version_major);
            // 16 MB dictionary is max due to 24 bit index  
        }
        else
        {
            //int on1_dictionary_mb = 1;
            //int on1_ht_bits = 17;
            int on1_dictionary_mb = 4;
            int on1_ht_bits = 19;
            //int on1_dictionary_mb = 8;
            //int on1_ht_bits = 19;
            //int on1_dictionary_mb = 16;
            //int on1_ht_bits = 20;
            
            TestOodleNetwork1UDPPacketCoder( 
                iob_packet_train,
                iob_packet_test,
                on1_dictionary_mb,
                on1_ht_bits ,
                for_oodle_version_major);
        }
    }
    //---------------------------
}


#ifdef __OODLE2X_H_INCLUDED__
static void Install_OodleX_Plugins_to_OodleNet()
{
    OodleNet_Plugins_SetPrintf(OodleXLog_Printf_Raw);
    OodleNet_Plugins_SetAssertion(OodleX_DisplayAssertion);
    OodleNet_Plugins_SetAllocators(OodleXMallocAligned,OodleXFree);
    OodleNet_Plugins_SetJobSystemAndCount(OodleX_CorePlugin_RunJob,OodleX_CorePlugin_WaitJob,OodleX_GetNumWorkerThreads());
}
#endif

extern "C" int main(int argc,char *argv[])
{
    #ifdef __OODLE2X_H_INCLUDED__
    // Init Oodle systems with default options :
    if ( ! OodleX_Init_Default(OODLE_HEADER_VERSION) )
    {
        fprintf(stderr,"OodleX_Init failed.\n");
        return 10;
    }
    Install_OodleX_Plugins_to_OodleNet();
    #endif

    OodleXLog_Printf_v1("example_packet [packet_file] [test_holdout] [for_oodle_version_major]\n");

    // defaults :
    const char * example_packet_file = c_example_packet_file;
    int example_packet_test_holdout_fraction_denominator = c_example_packet_test_holdout_fraction_denominator;
    int for_oodle_version_major = OODLE2NET_VERSION_MAJOR;

    // overrides in args :
    if ( argc >= 2 )
    {
        example_packet_file = argv[1];
    }
    if ( argc >= 3 )
    {
        example_packet_test_holdout_fraction_denominator = atoi(argv[2]);
        OodleXLog_Printf_v1("example_packet_test_holdout_fraction_denominator = %d\n",example_packet_test_holdout_fraction_denominator);
    }
    #ifndef BUILDING_TEST_PACKET
    if ( argc >= 4 )
    {
        for_oodle_version_major = atoi(argv[3]);
        OodleXLog_Printf_v1("for_oodle_version_major = %d\n",for_oodle_version_major);

        #ifdef USE_OODLE_LZ_DATA_COMPRESSION
        // set LZ back-compat version as well :
        
        struct OodleConfigValues oodleconfig;
        Oodle_GetConfigValues(&oodleconfig);
        oodleconfig.m_OodleLZ_BackwardsCompatible_MajorVersion = for_oodle_version_major;
        Oodle_SetConfigValues(&oodleconfig);
        #endif
    }
    #endif

    int ret = example_packet_onefile(example_packet_file,example_packet_test_holdout_fraction_denominator,for_oodle_version_major);

    #ifdef __OODLE2X_H_INCLUDED__
    //OodleX_Shutdown();
    OodleX_Shutdown(NULL,OodleX_Shutdown_LogLeaks_Yes,0);
    #endif
    
    //OodleXLog_Printf_v1("press a key\n");
    //fgetc(stdin);
    
    return ret;
}

//=================================================================

static void OodleLog_PacketCompression(OO_S64 tot_rawLen,OO_S64 tot_compLen,OO_S64 tot_numPackets)
{
    OodleXLog_Printf_v0("%.1f -> %.1f average = %.3f:1 = %.2f%% reduction\n",
                (double)tot_rawLen/tot_numPackets, (double)tot_compLen/tot_numPackets,
                (double)tot_rawLen/tot_compLen,(double)(tot_rawLen-tot_compLen)*100.0/tot_rawLen);
    OodleXLog_Printf_v1("%d packets; %lld -> %lld total bytes = %.2f%% of original\n",
        (int)tot_numPackets,(long long)tot_rawLen,(long long)tot_compLen,(double)tot_compLen*100.0/tot_rawLen);
}

//=================================================================

struct PacketCompressResult
{
    OO_S32 raw_size,compressed_size;
};

#if EXAMPLE_PACKET_DO_SORT

struct PacketCompressResult_Compare_raw_size
{
    bool operator () (const PacketCompressResult & lhs,const PacketCompressResult & rhs) const
    {
        return lhs.raw_size < rhs.raw_size;
    }
};

struct PacketCompressResult_Compare_compression_ratio
{
    bool operator () (const PacketCompressResult & lhs,const PacketCompressResult & rhs) const
    {
        // multiply could overflow OO_S32 so bump it up :
        return (OO_U64) lhs.compressed_size * rhs.raw_size < (OO_U64) rhs.compressed_size * lhs.raw_size;
    }
};

static void OodleLog_PacketCompressResults(PacketCompressResult * results, OO_SINTa count)
{
    OO_SINTa decile = count/10;

    if ( decile == 0 ) return;

    OodleXLog_Printf_v1("-------------------------------------------------------------\n");

    for(int rep=0;rep<2;rep++)
    {
        if ( rep == 0 )
        {
            OodleXLog_Printf_v1("Compression by size decile; %d packets per decile\n",(int)decile);
                
            // sort by raw size :
            std::sort(results,results+count,PacketCompressResult_Compare_raw_size());
        }
        else
        {
            OodleXLog_Printf_v1("Compression by ratio decile; %d packets per decile\n",(int)decile);
                
            // sort by raw size :
            std::sort(results,results+count,PacketCompressResult_Compare_compression_ratio());
        }
        
        for(int d=0;d<10;d++)
        {
            PacketCompressResult * base = results + decile * d;
            
            OO_SINTa decile_tot_raw_size = 0;
            OO_SINTa decile_tot_compressed_size = 0;
        
            for(OO_SINTa p=0;p<decile;p++)
            {
                decile_tot_raw_size += base[p].raw_size;
                decile_tot_compressed_size += base[p].compressed_size;
            }
        
            OodleXLog_Printf_v1("%d : %6.1f -> %6.1f = %.2f%% of original = %.2f%% reduction\n",
                d,
                (double)decile_tot_raw_size/decile,
                (double)decile_tot_compressed_size/decile,
                (double)decile_tot_compressed_size*100.0/decile_tot_raw_size,
                (double)(decile_tot_raw_size-decile_tot_compressed_size)*100.0/decile_tot_raw_size);
        }
    }

    OodleXLog_Printf_v1("-------------------------------------------------------------\n");
}

#else

static void OodleLog_PacketCompressResults(PacketCompressResult * results, OO_SINTa count)
{
    OOEX_UNUSED_VARIABLE(results);
    OOEX_UNUSED_VARIABLE(count);
}

#endif // EXAMPLE_PACKET_DO_SORT

//=================================================================

TCP variants of the OodleNetwork1 tests

//=================================================================

TestOodleNetwork_SelectDictionaryAndTrain :

fills out on1_shared, on1_initial_state and on1_dic

uses the provided io_packet_train training data to both make the dictionary and initialize the on1_initial_state

fills either tcp_initial_state or udp_initial_state or both. Set the one you don't need to NULL. on1_shared and on1_dic are always filled.


static void TestOodleNetwork_SelectDictionaryAndTrain( 
        OodleNetwork1_Shared * on1_shared,
        OodleNetwork1TCP_State * tcp_initial_state,
        OodleNetwork1UDP_State * udp_initial_state2,
        int on1_ht_bits,
        void * on1_dic, 
        OO_SINTa on1_dic_size,
        const OodleIOBuffer & iob_packet_train)
{
    if ( ! OodleNetwork1_SelectDictionarySupported() )
    {
        OOEX_ASSERT_FAILURE_ALWAYS("Can't build dictionary on embedded platforms.");
        return;
    }

    // parse the training set to vectors :

    const OO_U8 * packet_bin_buf = iob_packet_train.ptr;
    const OO_SINTa packet_bin_size = iob_packet_train.size;
    
    vector<const void *> dictionary_packet_pointers;
    vector<OO_S32> dictionary_packet_sizes;
    dictionary_packet_pointers.reserve(4096);
    dictionary_packet_sizes.reserve(4096);
    
    vector<const void *> dictest_packet_pointers;
    vector<OO_S32> dictest_packet_sizes;
    dictest_packet_pointers.reserve(4096);
    dictest_packet_sizes.reserve(4096);
    
    vector<const void *> training_packet_pointers;
    vector<OO_S32> training_packet_sizes;
    training_packet_pointers.reserve(4096);
    training_packet_sizes.reserve(4096);

    // get the number of channels ; first dword in packet bin : 
    const OO_U8 * packet_bin_ptr = packet_bin_buf;
    const OO_U8 * packet_bin_end = packet_bin_ptr + packet_bin_size;
    OO_S32 num_channels = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
    if ( num_channels == 0 ) num_channels = 1;
    OOEX_UNUSED_VARIABLE(num_channels);

    OO_SINTa training_tot_raw_len = 0;
    OO_SINTa dictionary_tot_raw_len = 0;
    OO_SINTa dictest_tot_raw_len = 0;
    
    while( packet_bin_ptr < packet_bin_end )
    {
        // grab the current packet header :
        OO_S32 channel = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OO_S32 bytes = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_ASSERT( channel >= 0 && channel < num_channels );
        OOEX_ASSERT( bytes > 0 );
        OOEX_UNUSED_VARIABLE(channel);
        
        if ( packet_bin_ptr + bytes > packet_bin_end )
        {
            // partial end packet
            break;
        }

        // put packet in one of three data sets :
        // semi-random :        
        int select = irandmod(3);
        
        if ( select == 0 )
        {
            training_packet_pointers.push_back( packet_bin_ptr );
            training_packet_sizes.push_back( bytes );
            training_tot_raw_len += bytes;
        }
        else if ( select == 1 )
        {
            dictionary_packet_pointers.push_back( packet_bin_ptr );
            dictionary_packet_sizes.push_back( bytes );
            dictionary_tot_raw_len += bytes;
        }
        else
        {
            dictest_packet_pointers.push_back( packet_bin_ptr );
            dictest_packet_sizes.push_back( bytes );
            dictest_tot_raw_len += bytes;
        }
                
        packet_bin_ptr += bytes;            
    }   
    OOEX_ASSERT( packet_bin_ptr == packet_bin_end );

    // 64M limit for dictest to keep speed reasonable
    while ( dictest_tot_raw_len > 64*1024*1024 )
    {
        // randomly move one from dictest to training :
        
        int dictest_packet_pointers_size = dictest_packet_pointers.size32();
        int i = irandmod(dictest_packet_pointers_size);
        const void * ptr = dictest_packet_pointers[i];
        OO_S32 size = dictest_packet_sizes[i];

        training_packet_pointers.push_back(ptr);
        training_packet_sizes.push_back(size);
        training_tot_raw_len += size;
        
        dictest_packet_pointers.erase_u(i);
        dictest_packet_sizes.erase_u(i);
        dictest_tot_raw_len -= size;
    }
            
    OO_S32 num_training_packets = training_packet_pointers.size32();
    OO_S32 num_dictionary_packets = dictionary_packet_pointers.size32();
    OO_S32 num_dictest_packets = dictest_packet_pointers.size32();
    
    if ( on1_dic_size > dictionary_tot_raw_len )
    {
        OodleXLog_Printf_v1("WARNING : training data not big enough for dictionary, clamping.\n");
    }
    
    on1_dic_size = OOEX_MIN(on1_dic_size,dictionary_tot_raw_len);
    
    OodleXLog_Printf_v1("clamped on1_dic_size = %d ; training len = %d\n",(int)on1_dic_size,(int)training_tot_raw_len);

    #if EXAMPLE_PACKET_OODLENETWORK_DO_TRIALS   
    //OO_S32 num_trials = 10;
    //OO_S32 num_generations = 2;
    OO_S32 num_trials = 3;
    OO_S32 num_generations = 1;
    double randomness = 200.0;
    #else
    OO_S32 num_trials = 1;
    OO_S32 num_generations = 0;
    double randomness = 0.0;
    #endif
    
    // dic_file_name (sdfp_dic.bin) is a cache to speed up repeated runs of this function
    //  only used if EXAMPLE_PACKET_OODLENETWORK_LOAD_EXISTING is enabled
    // it is *not* loaded by the runtime (that's the _runtimedata.bin)
    //  only loaded here
    const char * dic_file_name = EXAMPLE_PACKET_DEFAULT_OUTPUT_PATH "sdfp_dic.bin";
    // dic_file_name = NULL to not save
    //const char * dic_file_name = NULL;
    
    bool do_make_dic = true;
    bool do_load_existing = EXAMPLE_PACKET_OODLENETWORK_LOAD_EXISTING != 0;

    // On platforms where SelectDictionary funcs are not supported, force load existing
    if ( ! OodleNetwork1_SelectDictionarySupported() )
    {
        do_load_existing = true;
    }
    
    if ( do_load_existing && dic_file_name != NULL )
    {
        OodleXLog_Printf_v1("Trying to load : %s\n",dic_file_name);

        OO_SINTa dic_file_size;
        void * dic_file_buf = read_whole_file(dic_file_name,&dic_file_size);

        if ( dic_file_buf != NULL )
        {
            if ( on1_dic_size == (OO_SINTa)dic_file_size )
            {
                OodleXLog_Printf_v1("...got it!\n");
                memcpy(on1_dic,dic_file_buf,on1_dic_size);
                do_make_dic = false;
            }
            else
            {
                OodleXLog_Printf_v1("...wrong size!\n");
            }

            free(dic_file_buf);
        }
        else
        {
            OodleXLog_Printf_v1("...not found!\n");
        }
    }
    
    if ( do_make_dic )
    {   
        OodleNetwork1_SelectDictionaryFromPackets_Trials(on1_dic,(OO_S32)on1_dic_size,
                on1_ht_bits,
                dictionary_packet_pointers.data(),
                dictionary_packet_sizes.data(),num_dictionary_packets,
                
                dictest_packet_pointers.data(),
                dictest_packet_sizes.data(),num_dictest_packets,
                
                num_trials,randomness,num_generations);
        
        //----------------------
        // optionally write dictionary
        //  this isn't needed at runtime; the whole dictionary is stored in the "runtimedata" file as well

        if ( dic_file_name )
        {
            OodleXLog_Printf_v1("writing dictionary to: %s\n",dic_file_name);

            OO_BOOL write_ok = write_whole_file(dic_file_name,on1_dic,on1_dic_size);
            OOEX_ASSERT_ALWAYS( write_ok );
        }
    }
    
    if ( on1_shared )
        OodleNetwork1_Shared_SetWindow(on1_shared,on1_ht_bits,on1_dic,(OO_S32)on1_dic_size);
        
    // this is done by OodleNetwork1UDP_Train :
    //OodleNetwork1UDP_State_Reset(on1_initial_state);
        
    if ( tcp_initial_state )
    {
        OodleNetwork1TCP_Train(
                tcp_initial_state,
                on1_shared,
                training_packet_pointers.data(), 
                training_packet_sizes.data(),
                num_training_packets);
    }
    if ( udp_initial_state2 )
    {
        OodleNetwork1UDP_Train(
                udp_initial_state2,
                on1_shared,
                training_packet_pointers.data(), 
                training_packet_sizes.data(),
                num_training_packets);
    }
}

TestOodleNetwork1TCPPacketCoder_Transmission :

Test coding of iob_packet_test

Use the previously trained on1_shared and on1_initial_state

        
static void TestOodleNetwork1TCPPacketCoder_Transmission(
        const OodleNetwork1_Shared * on1_shared,
        const OodleNetwork1TCP_State * on1_initial_state,
        const OodleIOBuffer & iob_packet_test )
{

    //===========================================
    
    // point at the packet data :   
    const OO_U8 * packet_bin_buf = iob_packet_test.ptr;
    OO_SINTa packet_bin_size = iob_packet_test.size;
    
    // get the number of channels ; first dword in packet bin : 
    const OO_U8 * packet_bin_ptr = packet_bin_buf;
    const OO_U8 * packet_bin_end = packet_bin_ptr + packet_bin_size;
    OO_S32 num_channels = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
    
    //===========================================
    // initialize an OodleNetwork1 State for each channel :

    vector<OodleNetwork1TCP_State *> encoders;
    vector<OodleNetwork1TCP_State *> decoders;
    encoders.resize(num_channels);
    decoders.resize(num_channels);
    for (int c=0;c<num_channels;c++)
    {
        encoders[c] = (OodleNetwork1TCP_State *) malloc( OodleNetwork1TCP_State_Size() );
        decoders[c] = (OodleNetwork1TCP_State *) malloc( OodleNetwork1TCP_State_Size() );
        OodleNetwork1TCP_State_InitAsCopy( encoders[c], on1_initial_state );    
        OodleNetwork1TCP_State_InitAsCopy( decoders[c], on1_initial_state );    
    }
    
    //-------------------------------------------
    vector<PacketCompressResult> results;
    results.reserve(65536);
    
    //-------------------------------------------
    // make some buffers for compressed & decompressed data
    //  (we'll resize as needed)
    
    vector<OO_U8> compv; compv.reserve(65536);
    vector<OO_U8> decompv; decompv.reserve(65536);

    OO_S64 tot_rawLen = 0;
    OO_S64 tot_compLen = 0;
    OO_S64 tot_numPackets = 0;
        
    while( packet_bin_ptr < packet_bin_end )
    {
        // grab the current packet header :
        OO_S32 channel = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OO_S32 bytes = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_ASSERT( channel >= 0 && channel < num_channels );
        OOEX_ASSERT( bytes > 0 );
        
        if ( packet_bin_ptr + bytes > packet_bin_end )
        {
            // partial end packet
            break;
        }
        
        const OO_U8 * rawPtr = packet_bin_ptr;
        OO_SINTa curLen = bytes;
        
        compv.resize( OodleNetwork1_CompressedBufferSizeNeeded(curLen) );
        decompv.resize( curLen + OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN );
        
        OO_U8 * compPtr = compv.data();
        OO_U8 * decomp = decompv.data();

        OO_SINTa cur_compLen = 
            OodleNetwork1TCP_Encode(encoders[channel],on1_shared,rawPtr,curLen,compPtr);

        // check I'm not lying about compLen :
        compPtr[cur_compLen] = 0xCD;

        //OO_SINTa dec_compLen = 
        OodleNetwork1TCP_Decode(decoders[channel],on1_shared,compPtr,cur_compLen,
                    decomp,bytes);

        //OOEX_ASSERT_ALWAYS( cur_compLen == dec_compLen );     
        OOEX_ASSERT_ALWAYS( memcmp(packet_bin_ptr,decomp,bytes) == 0 );
        
        packet_bin_ptr += bytes;            
        tot_rawLen += bytes;
        tot_compLen += cur_compLen;
        tot_numPackets ++;
        
        results.push_back();
        results.back().raw_size = (OO_S32) bytes;
        results.back().compressed_size = (OO_S32) cur_compLen;
    }
    
    for (int c=0;c<num_channels;c++)
    {
        free(encoders[c]);
        free(decoders[c]);
    }
    
    encoders.release();
    decoders.release();
    
    OodleLog_PacketCompression(tot_rawLen,tot_compLen,tot_numPackets);
    
    OodleLog_PacketCompressResults(&results[0],results.size());
}
        
TestOodleNetwork1TCPPacketCoder

Test OodleNetwork1 on the packet data

does training, then compresses with the trained data

      
        
static void TestOodleNetwork1TCPPacketCoder( 
        const OodleIOBuffer & iob_packet_train ,
        const OodleIOBuffer & iob_packet_test ,
        int on1_dic_mb , int on1_ht_bits)
{
    OO_SINTa on1_dic_size = on1_dic_mb*1024*1024;

    on1_dic_size = OOEX_CLAMP(on1_dic_size,1024*1024,OODLENETWORK1_MAX_DICTIONARY_SIZE);
    
    OodleXLog_Printf_v1("OodleNetwork1 TCP models take : %d bytes per channel, %d bytes shared\n",
        (int) OodleNetwork1TCP_State_Size(),
        (int)(OodleNetwork1_Shared_Size(on1_ht_bits) + on1_dic_size));

    //-------------------------------------
    // OodleNetwork1 training requires a dictionary (on1_dic) of bytes to reference
    // The "on1_shared" object is static and shared by all channels
    //   it is built once from the on1_dic
    // The "on1_initial_state" is a trained initial state
    //   this state is copied into the per-channel states

    void * on1_dic = malloc( on1_dic_size );

    OodleNetwork1_Shared * on1_shared = (OodleNetwork1_Shared *) malloc( OodleNetwork1_Shared_Size(on1_ht_bits) );
    OOEX_ASSERT_ALWAYS( on1_shared != NULL );
    
    OodleNetwork1TCP_State * on1_initial_state = (OodleNetwork1TCP_State *) malloc( OodleNetwork1TCP_State_Size() );
    OOEX_ASSERT_ALWAYS( on1_initial_state != NULL );
    
    TestOodleNetwork_SelectDictionaryAndTrain( on1_shared, 
        on1_initial_state,NULL,
        on1_ht_bits, on1_dic, on1_dic_size, iob_packet_train );
    
    //-----------------------------------------------------
    
    OodleXLog_Printf_v1("OodleNetwork1 TCP [%d|%d] : ",on1_dic_mb,on1_ht_bits);
    
    TestOodleNetwork1TCPPacketCoder_Transmission( on1_shared, on1_initial_state, iob_packet_test );

    //-----------------------------------------------------

    free(on1_dic);
    free(on1_shared);
    free(on1_initial_state);
}
        
        
        
//=============================================================================

UDP variants of the OodleNetwork1 tests

//=============================================================================

TestOodleNetwork1UDPPacketCoder_Transmission :

Test coding of iob_packet_test

Use the previously trained on1_shared and on1_initial_state

        
static void TestOodleNetwork1UDPPacketCoder_Transmission(
        const OodleNetwork1_Shared * on1_shared,
        const OodleNetwork1UDP_State * state,
        const OodleIOBuffer & iob_packet_test )
{
    //-------------------------------------------
    vector<PacketCompressResult> results;
    results.reserve(65536);
    
    //-------------------------------------------
    // make some buffers for compressed & decompressed data
    //  (we'll resize as needed)
    
    OO_S64 tot_rawLen = 0;
    OO_S64 tot_compLen = 0;
    OO_S64 tot_numPackets = 0;
    
    // scratch space for just one packet compressed & decompressed
    vector<OO_U8> compv; compv.reserve(65536);
    vector<OO_U8> decompv; decompv.reserve(65536);
    
    //===========================================
    
    // point at the packet data :   
    const OO_U8 * packet_bin_buf = iob_packet_test.ptr;
    OO_SINTa packet_bin_size = iob_packet_test.size;
    
    // get the number of channels ; first dword in packet bin : 
    const OO_U8 * packet_bin_ptr = packet_bin_buf;
    const OO_U8 * packet_bin_end = packet_bin_ptr + packet_bin_size;
    OO_S32 num_channels = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
    if ( num_channels == 0 ) num_channels = 1;
    OOEX_UNUSED_VARIABLE(num_channels);

    //===========================================
            
    while( packet_bin_ptr < packet_bin_end )
    {
        // grab the current packet header :
        OO_S32 channel = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_UNUSED_VARIABLE(channel);
        OO_S32 bytes = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_ASSERT( channel >= 0 && channel < num_channels );
        OOEX_ASSERT( bytes > 0 );
        
        if ( packet_bin_ptr + bytes > packet_bin_end )
        {
            // partial end packet
            break;
        }
        
        const OO_U8 * rawPtr = packet_bin_ptr;
        OO_SINTa curLen = bytes;

        OO_SINTa comp_needed = OodleNetwork1_CompressedBufferSizeNeeded(curLen);
        if ( compv.size32() < comp_needed )
        {
            compv.resize(comp_needed);
        }
        
        // decomp must be sized to at least curLen + OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
        if ( decompv.size32() < curLen+OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN )
            decompv.resize(curLen+OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN);
        
        OO_U8 * comp = compv.data();
        OO_U8 * decomp = decompv.data();
    
        OO_SINTa cur_compLen = 
            OodleNetwork1UDP_Encode(state,on1_shared,rawPtr,curLen,comp);

        // check I'm not lying about compLen :
        comp[cur_compLen] = 0xCD;
        
        //OO_SINTa dec_compLen = 
        OO_BOOL ok = OodleNetwork1UDP_Decode(state,on1_shared,comp,cur_compLen,
                    decomp,bytes);
        OOEX_ASSERT_ALWAYS( ok );

        //OOEX_ASSERT_ALWAYS( cur_compLen == dec_compLen );     
        OOEX_ASSERT_ALWAYS( memcmp(packet_bin_ptr,decomp,bytes) == 0 );
                
        packet_bin_ptr += bytes;            
        tot_rawLen += bytes;
        tot_compLen += cur_compLen;
        tot_numPackets ++;
        
        results.push_back();
        results.back().raw_size = (OO_S32) bytes;
        results.back().compressed_size = (OO_S32) cur_compLen;
    }

    OodleLog_PacketCompression(tot_rawLen,tot_compLen,tot_numPackets);

    OodleLog_PacketCompressResults(&results[0],results.size());
}
        
TestOodleNetwork1UDPPacketCoder

Train the UDP coder then test using it

This function also demonstrated how you would compress the model and save it to disk

In real runtimes, you would just start from loading the compressed model

-----------------------------

OodleNetwork1_SavedModel_Header is an example header for the saved Oodle Network trained model.

It is not intended that you use this literally in your game. You should write your own IO code using your engine's IO systems to persist the network model.

Of note : the oodle_major_version is stored in this example header. If that is checked, then it forces the saved state to be regenerated whenever Oodle's version number is rev'ed. That is usually not necessary, old states will still work with most revs. So you may remove the check of oodle_major_version if you like.

NOTE : in real production code, your saved model should probably have encryption and/or a checksum applied for integrity checking and to prevent tampering.

      

struct OodleNetwork1_SavedModel_Header
{
    #define ON1_MAGIC   0x11235801
    OO_U32 magic;
    OO_U32 compressor;
    OO_U32 ht_bits;
    OO_U32 dic_size;
    OO_U32 oodle_major_version;
    OO_U32 dic_complen;
    OO_U32 statecompacted_size;
    OO_U32 statecompacted_complen;
};

OodleNetwork1_SavedModel_Header is written to file like a flat struct, but we ensure it's always little endian.

static void OodleNetwork1_SavedModel_Header_Read(OodleNetwork1_SavedModel_Header * pHeader,const void * from_memory)
{
    const OO_U32 * from_ptr = (const OO_U32 *)from_memory;
    pHeader->magic = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->compressor = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->ht_bits = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->dic_size = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->oodle_major_version = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->dic_complen = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->statecompacted_size = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->statecompacted_complen = OOEX_GET32_LE(from_ptr); from_ptr++;
    OOEX_ASSERT( (((OO_U8 *)from_ptr) - ((OO_U8 *)from_memory)) == sizeof(OodleNetwork1_SavedModel_Header) );
}

static void OodleNetwork1_SavedModel_Header_Write(const OodleNetwork1_SavedModel_Header * pHeader,void * to_memory)
{
    OO_U32 * to_ptr = (OO_U32 *)to_memory;
    OOEX_PUT32_LE(to_ptr,pHeader->magic); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->compressor); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->ht_bits); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->dic_size); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->oodle_major_version); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->dic_complen); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->statecompacted_size); to_ptr++;
    OOEX_PUT32_LE(to_ptr,pHeader->statecompacted_complen); to_ptr++;
    OOEX_ASSERT( (((OO_U8 *)to_ptr) - ((OO_U8 *)to_memory)) == sizeof(OodleNetwork1_SavedModel_Header) );
}


struct OodleNetwork1_Compressor
{
    void * dic;
    OodleNetwork1UDP_State * state;
    OodleNetwork1_Shared * shared;
};

static void OodleNetwork1_Compressor_Free(OodleNetwork1_Compressor * pCompressor)
{
    free(pCompressor->dic);
    free(pCompressor->shared);
    free(pCompressor->state);
}
    
static bool OodleNetwork1_Compressor_LoadFromFileData( OodleNetwork1_Compressor * pCompressor, const void * fileData, OO_SINTa fileSize )       
{
    // parse header :
    OodleNetwork1_SavedModel_Header header;
    OodleNetwork1_SavedModel_Header_Read(&header,fileData);
    
    if ( header.magic != ON1_MAGIC )
    {
        OodleXLog_Printf_v0("runtimedata_file ON1_MAGIC mismatch\n");
        return false;
    }
    
    // optional check for oodle_major_version being the same
    // NOTE : this is more conservative than necessary
    //  you may disable this check to keep loading old runtimedata files
    #if 0
    if ( header.oodle_major_version < 1 )
    {
        OodleXLog_Printf_v0("runtimedata_file version too old!\n");
        return false;
    }
    if ( header.oodle_major_version > OODLE2NET_VERSION_MAJOR )
    {
        OodleXLog_Printf_v0("runtimedata_file version newer that SDK!\n");
        return false;
    }
    #endif
    
    OO_S32 on1udpnew_ht_bits = header.ht_bits;
    OO_SINTa on1udpnew_dic_size = header.dic_size;
    
    OO_SINTa on1udpnew_dic_complen = header.dic_complen;
    OO_SINTa on1udpnew_statecompacted_size = header.statecompacted_size;
    OO_SINTa on1udpnew_statecompacted_complen = header.statecompacted_complen;
    
    // sanity checks on the header :
    // (in real use these should be if's and the header should be protected to
    //   forbid tampering)
    OOEX_ASSERT_ALWAYS( on1udpnew_dic_size >= on1udpnew_dic_complen );
    OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size+5 >= on1udpnew_statecompacted_complen );
    
    OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size > 0 && on1udpnew_statecompacted_size < OodleNetwork1UDP_StateCompacted_MaxSize() );
    
    OOEX_ASSERT_ALWAYS( fileSize == (OO_S64)sizeof(OodleNetwork1_SavedModel_Header) + on1udpnew_dic_complen + on1udpnew_statecompacted_complen );
    
    OodleXLog_Printf_v1("OodleNetwork1UDP Loading; dic comp %d , state %d->%d\n",
        (int)on1udpnew_dic_complen,(int)on1udpnew_statecompacted_size,(int)on1udpnew_statecompacted_complen);
    
    //-------------------------------------------
    // decompress on1udpnew_dic and on1udpnew_statecompacted

    pCompressor->dic = malloc(on1udpnew_dic_size);

    const void * on1udpnew_dic_comp_ptr = (const OO_U8 *)(fileData) + sizeof(OodleNetwork1_SavedModel_Header);

    OodleNetwork1UDP_StateCompacted * on1udpnew_compacted = (OodleNetwork1UDP_StateCompacted *) malloc( on1udpnew_statecompacted_size );

    void * on1udpnew_statecompacted_comp_ptr = (OO_U8 *)on1udpnew_dic_comp_ptr + on1udpnew_dic_complen;

    if ( header.compressor == (OO_U32)OodleLZ_Compressor_Invalid )
    {
        OOEX_ASSERT_ALWAYS( on1udpnew_dic_complen == on1udpnew_dic_size );

        memcpy(pCompressor->dic,on1udpnew_dic_comp_ptr,on1udpnew_dic_size); 

        OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size == on1udpnew_statecompacted_complen );

        memcpy(on1udpnew_compacted,on1udpnew_statecompacted_comp_ptr,on1udpnew_statecompacted_size);    
    }
    else
    {
        #ifdef USE_OODLE_LZ_DATA_COMPRESSION
        
        OO_SINTa decomp_dic_size = OodleLZ_Decompress(on1udpnew_dic_comp_ptr,on1udpnew_dic_complen,pCompressor->dic,on1udpnew_dic_size,OodleLZ_FuzzSafe_Yes);
    
        OOEX_ASSERT_ALWAYS( decomp_dic_size == on1udpnew_dic_size );

        OO_SINTa decomp_statecompacted_size = OodleLZ_Decompress(on1udpnew_statecompacted_comp_ptr,on1udpnew_statecompacted_complen,on1udpnew_compacted,on1udpnew_statecompacted_size,OodleLZ_FuzzSafe_Yes);

        OOEX_ASSERT_ALWAYS( decomp_statecompacted_size == on1udpnew_statecompacted_size );
        
        #else
        
        OOEX_ASSERT_FAILURE_ALWAYS("no USE_OODLE_LZ_DATA_COMPRESSION");
        
        #endif
    }
    
    //----------------------------------------------
    // Uncompact the "Compacted" state into a usable state

    OO_SINTa on1udpnew_state_size = OodleNetwork1UDP_State_Size();
    pCompressor->state = (OodleNetwork1UDP_State *) malloc( on1udpnew_state_size );
    OOEX_ASSERT_ALWAYS( pCompressor->state != NULL );

    if ( ! OodleNetwork1UDP_State_Uncompact_ForVersion(pCompressor->state,on1udpnew_compacted,header.oodle_major_version) )
    {
        OodleXLog_Printf_v0("OodleNetwork1UDP_State_Uncompact failed\n");
        return false;
    }
    free(on1udpnew_compacted);
    
    //----------------------------------------------
    // fill out on1udpnew_shared from the dictionary
            
    OO_SINTa shared_size = OodleNetwork1_Shared_Size(header.ht_bits);
    pCompressor->shared = (OodleNetwork1_Shared *) malloc( shared_size );
    OOEX_ASSERT_ALWAYS( pCompressor->shared != NULL );
    OodleNetwork1_Shared_SetWindow(pCompressor->shared,on1udpnew_ht_bits,pCompressor->dic,(OO_S32)on1udpnew_dic_size);
    
    return true;
}

static bool OodleNetwork1_Compressor_WriteToFile(const char * runtimedata_fileName, 
    const void * on1udpnew_dic,
    OO_SINTa on1udpnew_dic_size,
    const OodleNetwork1UDP_State *on1udpnew_state,
    OO_S32 on1udpnew_ht_bits,
    OodleLZ_Compressor file_compressor,
    int for_oodle_version_major)
{
    //-----------------------------------------------------
    // for saving the state to file,
    //  convert it to a "Compacted" state
    //  this is a smaller, non-runtime representation of the state

    OO_SINTa on1udpnew_statecompacted_maxsize = OodleNetwork1UDP_StateCompacted_MaxSize();

    OodleNetwork1UDP_StateCompacted * on1udpnew_compacted = (OodleNetwork1UDP_StateCompacted *) malloc( on1udpnew_statecompacted_maxsize );
    
    OO_SINTa on1udpnew_statecompacted_size = OodleNetwork1UDP_State_Compact_ForVersion(on1udpnew_compacted,on1udpnew_state,for_oodle_version_major);

    OO_SINTa on1udpnew_state_size = OodleNetwork1UDP_State_Size();
            
    OodleXLog_Printf_v1("OodleNetwork1UDP models compacted : %d bytes (from %d)\n",
        (int)on1udpnew_statecompacted_size,
        (int)on1udpnew_state_size);
    
    //-----------------------------------------------------
    // compress the block we will save
    //   we need to save on1udpnew_dic and on1udpnew_compacted
    // we can make the size of the saved state file even smaller by
    //   running normal OodleLZ compress on it
    
    OO_SINTa compressed_buffer_alloc_size = sizeof(OodleNetwork1_SavedModel_Header);
    #ifdef USE_OODLE_LZ_DATA_COMPRESSION
    compressed_buffer_alloc_size += OodleLZ_GetCompressedBufferSizeNeeded(file_compressor, on1udpnew_dic_size + on1udpnew_statecompacted_size );
    #else
    compressed_buffer_alloc_size += on1udpnew_dic_size + on1udpnew_statecompacted_size;
    #endif
    void * compressed_buffer = malloc( compressed_buffer_alloc_size );
    
    OO_U8 * comp_ptr = (OO_U8 *) compressed_buffer;
    
    // reserve space for header
    OO_U8 * header_pointer = comp_ptr;
    comp_ptr += sizeof(OodleNetwork1_SavedModel_Header);
    
    OO_SINTa on1udpnew_dic_complen;
    
    if (file_compressor == OodleLZ_Compressor_Invalid )
    {
        on1udpnew_dic_complen = on1udpnew_dic_size;
        memcpy(comp_ptr,on1udpnew_dic,on1udpnew_dic_size);
    }   
    else
    {
        #ifdef USE_OODLE_LZ_DATA_COMPRESSION
            
        // For real production you should use OodleLZ_CompressionLevel_Optimal2
        // for maximum compression without affecting decode speed
        //  (just slower to encode)
    
        OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Optimal2;
    
        on1udpnew_dic_complen = OodleXLZ_Compress_AsyncAndWait(OodleXAsyncSelect_All,file_compressor,on1udpnew_dic,on1udpnew_dic_size,comp_ptr,level);
        OOEX_ASSERT_ALWAYS( on1udpnew_dic_complen > 0 );
        
        #else
        
        on1udpnew_dic_complen = -1;
        OOEX_ASSERT_FAILURE_ALWAYS("no USE_OODLE_LZ_DATA_COMPRESSION");
        
        #endif
    }
    
    comp_ptr += on1udpnew_dic_complen;

    OO_SINTa on1udpnew_statecompacted_complen;
    
    if ( file_compressor == OodleLZ_Compressor_Invalid )
    {
        memcpy(comp_ptr,on1udpnew_compacted,on1udpnew_statecompacted_size);
        on1udpnew_statecompacted_complen = on1udpnew_statecompacted_size;
    }
    else
    {
        #ifdef USE_OODLE_LZ_DATA_COMPRESSION
        
        on1udpnew_statecompacted_complen = OodleLZ_Compress(file_compressor,on1udpnew_compacted,on1udpnew_statecompacted_size,comp_ptr,OodleLZ_CompressionLevel_Fast);

        OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_complen > 0 );
        
        #else
                
        OOEX_ASSERT_FAILURE_ALWAYS("no USE_OODLE_LZ_DATA_COMPRESSION");
        on1udpnew_statecompacted_complen = -1;
                
        #endif
    }
    
    comp_ptr += on1udpnew_statecompacted_complen;

    OO_SINTa total_complen = (OO_SINTa) ( comp_ptr - (OO_U8 *) compressed_buffer );
    
    OodleXLog_Printf_v1("OodleNetwork1UDP runtimedata file size : %d\n",(int)total_complen);
    
    free(on1udpnew_compacted); on1udpnew_compacted = NULL;
    
    //-----------------------------------------------------
    
    // fill the header :
    //  note : you should make a more robust & portable header for real use!
    
    OodleNetwork1_SavedModel_Header header = { 0 };
    header.magic = ON1_MAGIC;
    header.ht_bits = on1udpnew_ht_bits;
    header.compressor = (OO_U32) file_compressor;
    header.dic_size = (OO_U32) on1udpnew_dic_size;
    header.oodle_major_version = for_oodle_version_major;
    header.dic_complen = (OO_U32) on1udpnew_dic_complen;
    header.statecompacted_size = (OO_U32) on1udpnew_statecompacted_size;
    header.statecompacted_complen = (OO_U32) on1udpnew_statecompacted_complen;
    
    OodleNetwork1_SavedModel_Header_Write(&header,header_pointer); 
        
    //-----------------------------------------------------
    // write the compressed buffer containing on1udpnew_dic and on1udpnew_state to a file
    
    OodleXLog_Printf_v1("Writing runtimedata : %s\n",runtimedata_fileName);
    
    OO_BOOL write_ok = write_whole_file(runtimedata_fileName,compressed_buffer,total_complen);
    
    if ( ! write_ok )
    {
        OodleXLog_Printf_v1("ERROR : failed to write : %s\n",runtimedata_fileName);

        return false;
    }
    
    return true;
}
        
static void TestOodleNetwork1UDPPacketCoder( 
        const OodleIOBuffer & iob_packet_train ,
        const OodleIOBuffer & iob_packet_test ,
        int on1udpnew_dic_mb , int on1udpnew_ht_bits,
        int for_oodle_version_major)
{
    OO_SINTa on1udpnew_dic_size = on1udpnew_dic_mb*1024*1024;

    on1udpnew_dic_size = OOEX_CLAMP(on1udpnew_dic_size,1024*1024,OODLENETWORK1_MAX_DICTIONARY_SIZE);

    OO_SINTa shared_size = OodleNetwork1_Shared_Size(on1udpnew_ht_bits);
    OO_SINTa on1udpnew_state_size = OodleNetwork1UDP_State_Size();
                
    OodleXLog_Printf_v1("OodleNetwork1 UDP models take : %d bytes shared (%d + %d + %d)\n",
        (int) (on1udpnew_state_size + shared_size + on1udpnew_dic_size),
        (int)on1udpnew_state_size,(int) shared_size,(int) on1udpnew_dic_size
        );
                        
    // this is all the data loaded by the runtime
    //  (see example_network_client.cpp)
    const char * runtimedata_fileName = EXAMPLE_PACKET_DEFAULT_OUTPUT_PATH "example_packet_on1udpnew_runtimedata.bin";

    //-------------------------------------
    // train the dictionary and on1udpnew state
    //  this is done offline and the result is saved to a file

    #if EXAMPLE_PACKET_OODLENETWORK_LOAD_EXISTING
    // skip training and just load the file we made last time
    OOEX_UNUSED_VARIABLE(iob_packet_train);
    #else
    if ( 1 )
    {
        // OodleNetwork1UDP training requires a dictionary (on1udpnew_dic) of bytes to reference
        // The "on1udpnew_shared" object is static and shared by all channels
        //   it is built once from the on1udpnew_dic
        // The "on1udpnew_initial_state" is a trained initial state
        //   this state is copied into the per-channel states

        void * on1udpnew_dic = malloc( on1udpnew_dic_size );
        
        OodleNetwork1_Shared * on1udpnew_shared = (OodleNetwork1_Shared *) malloc( shared_size );
        OOEX_ASSERT_ALWAYS( on1udpnew_shared != NULL );
        
        OodleNetwork1UDP_State * on1udpnew_state = (OodleNetwork1UDP_State *) malloc( on1udpnew_state_size );
        OOEX_ASSERT_ALWAYS( on1udpnew_state != NULL );
        
        TestOodleNetwork_SelectDictionaryAndTrain(
            on1udpnew_shared, 
            NULL,on1udpnew_state,
            on1udpnew_ht_bits, on1udpnew_dic, on1udpnew_dic_size, iob_packet_train );
        
        // on1udpnew_shared is NOT written, it must be remade on load
        free(on1udpnew_shared); on1udpnew_shared = NULL;

        //-----------------------------------------------------
        
        OodleXLog_Printf_v1("Saving state...\n");

        #ifdef USE_OODLE_LZ_DATA_COMPRESSION
        
        // write the state compressed :
        OodleLZ_Compressor file_compressor = OodleLZ_Compressor_Kraken;

        OOEX_ASSERT( OodleLZ_Compressor_CanDecodeFuzzSafe(file_compressor) );

        #else
        
        // uncompressed state :
        
        OodleLZ_Compressor file_compressor = OodleLZ_Compressor_Invalid;
        
        #endif

        bool ok =OodleNetwork1_Compressor_WriteToFile(runtimedata_fileName,on1udpnew_dic,on1udpnew_dic_size,on1udpnew_state,on1udpnew_ht_bits,file_compressor,for_oodle_version_major);

        free(on1udpnew_dic);
        free(on1udpnew_state);
        
        if (! ok )
        {
            OodleXLog_Printf_v0("runtimedata save failed\n");
            OOEX_ASSERT_FAILURE_ALWAYS("fail");
        }
    }
    #endif // EXAMPLE_PACKET_OODLENETWORK_LOAD_EXISTING     
    
    // real runtime would start here
    // trained model is already compressed and saved to disk
    
    //-----------------------------------------------------
    // load the saved runtimedata and use it for compression
    if ( 1 )
    {
        // read the compressed runtimedata file
        OO_SINTa runtimedata_fileSizeA = 0;
        void * runtimedata_fileData = read_whole_file(runtimedata_fileName,&runtimedata_fileSizeA);
        OOEX_ASSERT_ALWAYS( runtimedata_fileData != NULL );
        
        if ( runtimedata_fileSizeA < 4096 )
        {
            OodleXLog_Printf_v0("runtimedata_fileSize too small\n");
            OOEX_ASSERT_FAILURE_ALWAYS("fail");
        }

        OodleNetwork1_Compressor compressor = { 0 };

        bool ok = OodleNetwork1_Compressor_LoadFromFileData(&compressor,runtimedata_fileData,runtimedata_fileSizeA);

        free(runtimedata_fileData); runtimedata_fileData = NULL;
    
        if (! ok )
        {
            OodleXLog_Printf_v0("runtimedata load failed\n");
            OOEX_ASSERT_FAILURE_ALWAYS("fail");
        }
    
        //----------------------------------------------------- 
        // our runtime data structures are now loaded
        // test compressing some packets
        
        OodleXLog_Printf_v0("OodleNetwork1 UDP [%d|%d] : ",on1udpnew_dic_mb,on1udpnew_ht_bits);
        
        TestOodleNetwork1UDPPacketCoder_Transmission( compressor.shared, compressor.state, iob_packet_test );

        //-----------------------------------------------------

        OodleNetwork1_Compressor_Free(&compressor);
    }
    //----------------------------------------------------- 
}
        
//=================================================

A simple random number generator :

Marsaglia's KISS99


static OO_U32 x = 123456789,y = 362436000,z = 521288629,c = 7654321; // seeds

static OO_U32 KISS99()
{
    x = 69069*x+12345;  
    
    y ^= (y<<13);
    y ^= (y>>17); 
    y ^= (y<<5);  

    static const OO_U64 a = 698769069ULL;       
    OO_U64 t = a*z+c; 
    c = (OO_U32)(t>>32); 
    z = (OO_U32)t;
    
    return (x+y+z);
}

// returns a value in [0,size-1]
static OO_U32 irandmod(OO_U32 size)
{
    // use mul hi
    return (OO_U32)((KISS99() * (OO_U64)size) >> 32);
}


 
example_network_client : Example with simple network client supportExamples 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle Ext
Navigation
 OodleX Utils
 Oodle2 Ext API Documentation
 Welcome to Oodle
 Change Log

Oodle Ext provides threading and IO helpers to run compression tasks multi-threaded. You don't need to use them, you can just call the simple single threaded functions.

Functions in Oodle Ext start with OodleX_.

OodleX requires a single Init and Shutdown call per application run.

See Oodle2 Core vs Oodle2 Ext

Oodle async operations are coordinated through the OodleHandle (see About OodleXHandle) which provides a way to check on async work or IO's, or groups or chains of work and IO's.

It is recommended that you ship your game using Oodle Core only. Oodle Ext is intended for tools and to get a quick start using Oodle. Some console & mobile platforms versions of Oodle do not include Oodle Ext.

Naming Conventions

OodleX tries to make it clear what its functions do through a few naming conventions.

Functions with _Async on the name start an async operation. Generally these functions return immediately, but the actual operation is not done yet. Generally they return an OodleHandle.

Functions with _Wait on the name wait on an async operation. They may block a long time, depending on the async operation. Typically they also free the operation or provide a flag for whether to free it.

Generally for each _Async function you want to pair it with a _Wait. There are function-specific pairings, and those are generally indicated by a shared prefix in the function name, such as OodleXLZ_Compress_Async then OodleXLZ_Compress_Wait_GetResult. If there is not a specific pair you can always use OodleX_Wait on any OodleHandle.

Functions with _AsyncAndWait in the name indicate that they fire off an async op and then wait on it. This is different than a synchronous function (sometimes indicated with _Sync in the name, but more often just any function that doesn't have Async in the name); a synchronous function runs on the calling thread, whereas an _AsyncAndWait function is fired off to some async runner and then blocked on.

Some functions have Wide or Narrow in the name. Narrow means the task is run asynchronously, but is run as a single non-separable operation. Wide means the task can be broken into several smaller pieces and the pieces can be run simultaneously using several threads. These correspond to flags of OodleXAsyncSelect.

Functions generally have subsystem prefixes to indicate what kinds of data they work on. For example OodleWork_ prefix functions can only work on OodleHandles that correspond to Work objects. It's up to you to keep handles in the correct subsytem. The generic Oodle_ prefix functions (like OodleX_WaitAll or OodleX_GetStatus) work on any subsystem.

There are lots of enum arguments in Oodle. The values that the enums can take always start with the type of the enum; eg. OodleXCopyFileFlags has values like OodleXCopyFileFlags_Overwrite and OodleXCopyFileFlags_DontOverwriteExisting. One special common case is enums that describe a boolean choice; in that case the name is always the enum type + _Yes or _No , as in OodleXHandleDeleteIfDone which has values OodleXHandleDeleteIfDone_Yes and OodleXHandleDeleteIfDone_No.
 

OODLEX_PATH_DELIMOodle2 Ext API Documentation 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Wait_GetInfo
Navigation
 OodleXIOQ_GetInfo
 OodleXIOQ_GetLastPendingOpOnFile
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_Wait_GetInfo( OodleXIOQFile file,
                                OodleXFileInfo * pInto,
                                OO_S32 * pAlignmentRequired OODEFAULT( NULL ) );
Discussion
Get Info about a file ; if the file is not open yet, wait for it
Parameters
file  the IOQFile to query
pInto  filled with OodleXFileInfo
pAlignmentRequired  (optional) filled with alignment required
Return Value
return  true if successful; if GetInfo returns false, pInto and pAlignmentRequired are untouched.
Discussion

This function is like OodleXIOQ_GetInfo , but will not return false if the Open operation is still pending; instead it will block the calling thread until the Open is done so that info is available.


 

OodleXIOQ_GetInfoOodleX low level async ioOodleXIOQ_GetLastPendingOpOnFile

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_MALLOC_MINIMUM_ALIGNMENT
Navigation
 OODLE_JOB_MAX_DEPENDENCIES
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#if defined(__RAD64__)
#define OODLE_MALLOC_MINIMUM_ALIGNMENT 16
#else
#define OODLE_MALLOC_MINIMUM_ALIGNMENT 8
#endif
Description
OodleMallocAligned will be asked to provide at least OODLE_MALLOC_MINIMUM_ALIGNMENT

OODLE_MALLOC_MINIMUM_ALIGNMENT is the size of two pointers.


 

Core BaseCore BaseOODLE_JOB_MAX_DEPENDENCIES

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How do I limit the encoder memory use?
Navigation
 FAQ: How do I use Oodle with no allocator?
 Frequently Asked Questions
 FAQ: How much memory do the Oodle compressors use ?
 Welcome to Oodle
 Change Log

If you wish to limit the encoder memory use, for runtime encoding :

First of all use a compression level in the "fast/normal" range, not an optimal. The optimal levels use a lot of memory, will call out to your installed malloc plugin (not use scratch), and is hard to limit. The optimal levels are not intended for runtime compression.

You can pass in preallocated scratch memory to OodleLZ_Compress using scratchMem and scratchSize arguments. The encoder will take memory from this pool first before calling to your installed allocator.

You can install your own allocator with OodleCore_Plugins_SetAllocators , and then log the allocations or make it an error if OodleLZ_Compress tries to use the allocator. That way you can ensure that OodleLZ_Compress only takes memory from the pool you pass in.

Okay, so now how much scratch memory to pass in?

It depends on the OodleLZ_CompressOptions and OodleLZ_CompressionLevel and OodleLZ_Compressor, as well as the length of the buffer you compress.

You can call OodleLZ_GetCompressScratchMemBound to get a bound on how much memory will be used. If you provide that much scratch, Oodle will not need to allocate.

OodleLZ_GetCompressScratchMemBound may return OODLELZ_SCRATCH_MEM_NO_BOUND which means it cannot provide a strict bound.

By default, memory use grows roughly proportionally to buffer size. If you want to force a lower maximum memory use, you can modify the OodleLZ_CompressOptions:matchTableSizeLog2 setting. The majority of scratch memory use comes from 4 bytes per matchTableSizeLog2 ; eg if you set matchTableSizeLog2 = 20 , that would use 4 MB for the "match table".

Other than the match table, about 1 MB of additional scratch is needed by the encoder (for unbounded length input buffers; smaller inputs need less), so at matchTableSizeLog2 = 20 , you would need about 5 MB, 4 MB from the match table and 1 MB for other.
 

FAQ: How do I use Oodle with no allocator?Frequently Asked QuestionsFAQ: How much memory do the Oodle compressors use ?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetInfo
Navigation
 OodleXIOQ_Wait
 OodleXIOQ_Wait_GetInfo
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_GetInfo( OodleXIOQFile file,
                           OodleXFileInfo * pInto,
                           OO_S32 * pAlignmentRequired OODEFAULT( NULL ) );
Discussion
Get Info about a file
Parameters
file  the IOQFile to query
pInto  filled with OodleXFileInfo
pAlignmentRequired  (optional) filled with alignment required
Return Value
return  true if successful; if GetInfo returns false, pInto and pAlignmentRequired are untouched.
Discussion

If the file is not yet open, GetInfo will fail and return false. eg. if OodleXIOQ_OpenForRead_Async has been done but the request is still pending.

If the file size can not be queried it is set to OODLEX_FILE_SIZE_INVALID.

If pAlignmentRequired is given, it is filled with the alignment required to use this file. OODLEX_IO_MAX_ALIGNMENT is guaranteed to always be okay, so if you align to that then you are fine. See About OodleIOQ for more about alignment.


 

OodleXIOQ_WaitOodleX low level async ioOodleXIOQ_Wait_GetInfo

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz_threadphased : Example of 2-thread ThreadPhased decoding
Navigation
 example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
 Examples
 example_network_client : Example with simple network client support
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz_threadphased

This example demonstrates the ability of Kraken to decode using "ThreadPhased" parallelism

See About OodleLZ ThreadPhased Decode for details.

ThreadPhased decoding provides a 1X-2X speedup, typically around 33%-50%

This example implements an entire ThreadPhased decoder, in example_lz_threadphased_decompress.

The idea is you may take this example code and modify it to run in your own threading or job system, so that you may implement the threading however you look.

example_lz_threadphased_decompress can be used without OodleX or the Oodle Worker system.

In this example I use OodleX_CreateThread and OodleX_Semaphore ; these are just handy cross-platform implementations for me. The intention is that you replace them with your own threading system calls.

For the semaphore, it's important that it tries to avoid going into an OS Wait (thread sleep) when the two threads are nearly synchronized. To avoid this you want a user-space spin-backoff loop to try to keep the two threads awake together. One option is to use something like "fastsemaphore" as a wrapper to your underlying OS semaphore :

http://cbloomrants.blogspot.com/2011/12/12-08-11-some-semaphores.html

If you use that "fastsemaphore" make sure to set int spin_count = 1; // ! set this for your system something like 100 is usually reasonable, and probably add a backoff.


#include "../include/oodle2x.h"
#include "ooex.h"

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "make_example_input.h"

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz_threadphased
#endif

//===========================================================
// file names :

static const char * in_name = "oodle_example_input_file";

//===========================================================

static bool example_lz_threadphased_decompress(void * comp_buf,OO_SINTa comp_len,void * dec_buf,OO_SINTa dec_size,bool async);
    
//===========================================================

extern "C" int main(int argc,char *argv[])
{
    // Init Oodle systems with default options :
    if ( ! OodleX_Init_Default(OODLE_HEADER_VERSION) )
    {
        fprintf(stderr,"OodleX_Init failed.\n");
        return 10;
    }   

    if ( argc >= 2 ) in_name = argv[1];
    else make_example_input(in_name);

    OodleXLog_Printf_v1("example_lz_threadphased : %s\n",in_name);

    // read the input file to the global buffer :
    OO_S64 in_size_64;
    void * in_buffer = OodleXIOQ_ReadMallocWholeFile_AsyncAndWait(in_name,&in_size_64);
    
    if ( ! in_buffer)
    {
        OodleXLog_Printf_v0("failed to read %s\n",in_name); 
        return 10;
    }
    
    OO_SINTa in_size = OodleX_S64_to_SINTa_check( in_size_64 );
    
    //=========================================
    // select options :
    
    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    //OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Normal;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Fast;

    // this example is only valid for compressors that support ThreadPhased decode : (eg. Kraken)
    OOEX_ASSERT( OodleLZ_Compressor_CanDecodeThreadPhased(compressor) );

    //=========================================
    
    // allocate memory big enough for compressed data :
    void * comp_buf = OodleXMalloc( OodleLZ_GetCompressedBufferSizeNeeded(compressor,in_size) );    
    // memory to decode to :
    void * dec_buf = OodleXMalloc( in_size );
    
    //=========================================
    // compress the input :
    
    // compress :
    // this is just a normal whole-block compress, no special parallel mode is needed
    OO_SINTa comp_len = OodleLZ_Compress(compressor,in_buffer,in_size,comp_buf,level);
    
    OodleXLog_Printf_v1("Compressed : %d -> %d\n",(int)in_size,(int)comp_len);
    
    //=======================================
    
We can decompress asynchronously using the "ThreadPhased" helper in OodleX.

The Narrow helper uses 2 threads and frees up the calling thread to do other work.

OodleXLZ_Decompress_ThreadPhased_Narrow_Async uses the OodleX Worker thread system, which is started by default in OodleX_Init.

NOTE for maximum speed you should pass in the scratch space needed by OodleXLZ_Decompress_ThreadPhased_Narrow_Async pre-allocated, so it doesn't have to do the allocation internally.

    
    OodleXLog_Printf_v1("OodleXLZ_Decompress_ThreadPhased_Narrow_Async...\n");
    
    OodleXHandle decomp_handle = OodleXLZ_Decompress_ThreadPhased_Narrow_Async(comp_buf,comp_len,dec_buf,in_size);
    
    // .. can do other work on the main thread now ..
    
    OodleXStatus decomp_status = OodleX_WaitAndDelete(decomp_handle);
    if ( decomp_status == OodleXStatus_Error )
    {
        OodleXLog_Printf_v1("Error!\n");
    }
    
    // check it :
    OOEX_ASSERT( memcmp(in_buffer,dec_buf,in_size) == 0 );
    
    //=======================================
    // do our own thread-phased decode :
        
    OodleXLog_Printf_v1("example_lz_threadphased_decompress ");

    // run a few reps to stress test :  
    for(int rep=0;rep<10;rep++)
    {
        // run async and sync options :
        for(int async = 0;async<=1;async++)
        {
            OodleXLog_Printf_v1(async ? "+" : "-");
        
            // wipe out dec_buf to make sure we decode correctly :
            memset(dec_buf,0xEE,in_size);
        
            if ( !  example_lz_threadphased_decompress(comp_buf,comp_len,dec_buf,in_size,!!async) )
            {
                OodleXLog_Printf_v1("Error!\n");
            }
        
            // check it :
            OOEX_ASSERT( memcmp(in_buffer,dec_buf,in_size) == 0 );
        }
    }
    OodleXLog_Printf_v1("\n");
    
    //=======================================
    
    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
    OodleXFree_IOAligned(in_buffer);

    OodleX_Shutdown(NULL,OodleX_Shutdown_LogLeaks_Yes,0);
    
    return 0;
}

//===========================================================
**

example_lz_threadphased_threadfunc

Example of how to run a ThreadPhased decode yourself.

The basic idea of ThreadPhased decoding is that the OodleLZ_Decompress work on each BLOCK can be split into two phases. This can be invoked by just calling OodleLZ_Decompress twice on the same block, first with OodleLZ_Decode_ThreadPhase1, then with OodleLZ_Decode_ThreadPhase2.

To get parallelism, we can run the two phases on two separate threads.

The rule is that you must run the Phase2 on each block after the Phase1 for that block is done, and with the same "decoderMem" pointer. The Phase2 decodes on all blocks must be done in sequential order (unless they are Seek Resets). The decoder memory used for OodleLZ_Decompress here must be larger than normal, of size OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded().

Our thread model here is a two-thread circular buffer scan with semaphore signalling.

Thread 1 does :

For each block Wait on sem_blocksavail to get an available circular buffer slot Do Phase1 Decompress into a slot Post sem_phase1done

Thread 2 does :

For each block Wait on sem_phase1done to get a slot with phase1 decode done Do Phase2 Decompress on that slot Post sem_blocksavail to signal Thread 1 that this slot may be reused

**


struct example_lz_threadphased_threaddata
{
    volatile OO_U32 * error_cancel;
    OodleX_Semaphore * sem_consume;
    OodleX_Semaphore * sem_produce;
    
    OO_SINTa num_blocks;
    OO_SINTa num_scratch_blocks;
    OO_U8 * scratch_mem;
    OO_SINTa scratch_block_size;
    
    OO_U8 * rawBuf; OO_SINTa rawSize;
    const OO_U8 * compBuf; OO_SINTa compSize;
    OodleLZ_Decode_ThreadPhase threadPhase;
    
    bool success;
};

#define THREAD_ERROR    0
#define THREAD_SUCCESS  1

// thread_function_DecodePhase
//  can be used for both Phase1 and Phase2 !
static OO_U32 OODLE_CALLBACK example_lz_threadphased_threadfunc( void * user_data )
{
    example_lz_threadphased_threaddata * data = (example_lz_threadphased_threaddata *)user_data;
    
    const OO_U8 * compPtr = (OO_U8 *)(data->compBuf);
    const OO_U8 * compEnd = compPtr + data->compSize;
    
    OO_SINTa decoderMemSize = data->scratch_block_size;
        
    OO_SINTa scratch_i = 0;
    for(OO_SINTa block_pos = 0;block_pos < data->rawSize;block_pos += OODLELZ_BLOCK_LEN, scratch_i++)
    {
        // consume one :
        OodleX_Semaphore_Wait(data->sem_consume);
    
        // relaxed load of shared error_cancel variable ; Sem Wait acts as Acquire barrier
        if ( *(data->error_cancel) ) return THREAD_ERROR;
    
        if ( scratch_i == data->num_scratch_blocks ) scratch_i = 0;
        
        OO_U8 * decoderMem = data->scratch_mem + scratch_i * decoderMemSize;
        
        OO_U8 * chunk_ptr = (OO_U8 *)(data->rawBuf) + block_pos;
                    
        OO_SINTa block_len = OOEX_MIN((data->rawSize - block_pos),OODLELZ_BLOCK_LEN);
                    
        OO_BOOL indy;
        OO_SINTa block_complen = OodleLZ_GetCompressedStepForRawStep(compPtr,compEnd-compPtr,block_pos,block_len,NULL,&indy);
        if ( block_complen <= 0 )
        {
            // handle error
            // relaxed store of shared variable; Sem Post acts as release barrier
            *(data->error_cancel) = 1;
            OodleX_Semaphore_Post(data->sem_produce);
            return THREAD_ERROR;
        }
                    
        OO_SINTa gotLen = OodleLZ_Decompress(compPtr,block_complen,chunk_ptr,block_len,
            OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,
            data->rawBuf,data->rawSize,
            NULL,NULL,
            decoderMem,decoderMemSize,
            data->threadPhase
            );
        OOEX_ASSERT( gotLen == block_len+block_pos );
        if ( gotLen != block_len+block_pos )
        {
            // handle error
            // relaxed store of shared variable; Sem Post acts as release barrier
            *(data->error_cancel) = 1;
            OodleX_Semaphore_Post(data->sem_produce);
            return THREAD_ERROR;
        }
        
        compPtr += block_complen;
        
        OodleX_Semaphore_Post(data->sem_produce);
    }

    data->success = true;   
    return THREAD_SUCCESS;
}

static bool example_lz_threadphased_decompress(void * comp_buf,OO_SINTa comp_len,void * dec_buf,OO_SINTa dec_size,bool async)
{
    
*

circularBufferBlockCount is the number of circular buffer slots for the two threads to communicate through

circularBufferBlockCount >= 2 higher is faster because it allows less synchronization of the two threads lower means less memory required circularBufferBlockCount >= 4 is reasonable circularBufferBlockCount >= 6 is close to full speed

*

    OO_SINTa circularBufferBlockCount = 6; // parameter

    // async is an option for whether the whole operation is run async off this thread or not
    //  (eg. with 2 additional threads or 1 additional thread)

    // check that the data contains a valid ThreadPhased compressor :
    OodleLZ_Compressor compressor = OodleLZ_GetAllChunksCompressor(comp_buf,comp_len,dec_size);
    if (! OodleLZ_Compressor_CanDecodeThreadPhased(compressor) )
    {
        OodleXLog_Printf_v1("Asked for ThreadPhase decode but ! OodleLZ_Compressor_CanDecodeThreadPhased\n");
        return false;
    }

    // count the number of OODLELZ_BLOCK_LEN in the total size :
    OO_SINTa nBlocks = (dec_size + OODLELZ_BLOCK_LEN-1)/OODLELZ_BLOCK_LEN;

    // don't need more than nBlocks :
    circularBufferBlockCount = OOEX_MIN(circularBufferBlockCount,nBlocks);

    // allocate space for the scratch circular buffer :
    OO_SINTa scratchBlockSize = OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded();
    OO_SINTa scratchBufSize = scratchBlockSize * circularBufferBlockCount;
    
    // NOTE in production you may wish to preallocate this memory
    void * scratchBuf = OodleXMalloc(scratchBufSize);
    
    //=========================================================
    // set up the data needed for the thread phases
    // NOTE if you want to make the whole decode asynchronous,
    //   you can't put this on the stack, you need to package it up in memory
    
    // OodleX_Semaphore just initialize with 0 :
    OodleX_Semaphore sem_blocksavail = 0;
    OodleX_Semaphore sem_phase1done = 0;
    OO_U32 shared_error_cancel = 0; // shared atomic variable
    
    // starting state is that all circular buffer slots are available :
    OodleX_Semaphore_Post(&sem_blocksavail,(OO_S32)circularBufferBlockCount);
    
    example_lz_threadphased_threaddata td1 = { 0 };
    
    td1.error_cancel = &shared_error_cancel;
    td1.success = false;
    td1.compBuf = (OO_U8 *)(comp_buf);
    td1.compSize = comp_len;
    td1.rawBuf = (OO_U8 *)(dec_buf);
    td1.rawSize = dec_size;
    td1.num_blocks = nBlocks;
    td1.num_scratch_blocks = circularBufferBlockCount;
    td1.scratch_mem = (OO_U8 *)(scratchBuf);
    td1.scratch_block_size = scratchBlockSize;
        
    // thread 1 waits for blocks to be available in the circular buffer
    //  and posts that phase1 is done
    td1.threadPhase = OodleLZ_Decode_ThreadPhase1;
    td1.sem_consume = &sem_blocksavail;
    td1.sem_produce = &sem_phase1done;
    
    // thread 2 waits for each block to reach phase1done
    //  and then posts that the block is reusable
    //  same as thread 1, just swap the semaphores :
    example_lz_threadphased_threaddata td2;
    td2 = td1;
    td2.threadPhase = OodleLZ_Decode_ThreadPhase2;
    td2.sem_consume = &sem_phase1done;
    td2.sem_produce = &sem_blocksavail;
    
    // create a thread to run Phase1 :
    // NOTE : in production you probably don't want to create a thread every time you make
    //  this decompress call.  Rather use an idle thread that's already created.
    OodleX_Thread thread1 = OodleX_CreateThread(example_lz_threadphased_threadfunc,&td1);
    
    // either run Phase2 asynchronously (on another thread) or synchronously on this thread :
    if ( async )
    {
        OodleX_Thread thread2 = OodleX_CreateThread(example_lz_threadphased_threadfunc,&td2);
    
        // ... current thread is now available while decompress runs on 2 other threads ...
        // ... return and do other work ...
    
        OodleX_WaitAndDestroyThread(thread2);
    }
    else
    {
        // synchronous version - just run Phase2 on this thread :
        example_lz_threadphased_threadfunc(&td2);
    }
    
    OodleX_WaitAndDestroyThread(thread1);
    
    // OodleX_Semaphore doesn't need cleanup
    
    //===========================================================
    
    OodleXFree(scratchBuf);
    
    bool ok = td1.success && td2.success;
    
    return ok;
}


 
example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle CoreExamplesexample_network_client : Example with simple network client support

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_WriteWholeFile_AsyncAndWait
Navigation
 OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
 OodleXIOQ_CopyFile_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_WriteWholeFile_AsyncAndWait( const char * vfsName,
                                               const void * buffer,
                                               OO_SINTa size,
                                               OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ) );
Discussion
See OodleXIOQ_OpenWriteWholeFileClose_Async
 
OodleXIOQ_ReadMallocWholeFile_AsyncAndWaitOodleX low level async ioOodleXIOQ_CopyFile_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Core Base
Navigation
 Oodle2 Core API Documentation
 Core plugins
 Welcome to Oodle
 Change Log
  •  Defines
    •  OODLE_MALLOC_MINIMUM_ALIGNMENT
    •  OODLE_JOB_MAX_DEPENDENCIES
    •  OODLE_JOB_NULL_HANDLE
    •  t_fp_Oodle_Job
    •  OODLE_HEADER_VERSION
    •  OodleNetworkVersion
  •  Enumerants
    •  Oodle_UsageWarnings
  •  Structures
    •  OodleConfigValues
  •  Functions
    •  Oodle_GetConfigValues
    •  Oodle_SetConfigValues
    •  Oodle_SetUsageWarnings
    •  Oodle_CheckVersion
    •  Oodle_LogHeader
  •  Typedefs
    •  t_OodleFPVoidVoid
    •  t_OodleFPVoidVoidStar

 
Oodle2 Core API DocumentationOodle2 Core API DocumentationOODLE_MALLOC_MINIMUM_ALIGNMENT

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenForWriteTempName_Async
Navigation
 OodleXIOQ_OpenForWriteCreate_Async
 OodleXIOQ_CloseFile_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenForWriteTempName_Async( OodleXIOQFile * pFile,
                                                   const char * nameBase OODEFAULT( NULL ),
                                                   OO_S64 initialFileSize OODEFAULT( OODLEX_FILE_OPEN_NO_RESERVE_SIZE ),
                                                   OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                   const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                   OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                   OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                   const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                   OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start opening a file for write
Parameters
pFile  filled with a handle to the file which will be opened
nameBase  (optional) prefix of the temp file name that will be written (VFS, UTF-8)
initialFileSize  (optional) pre-allocate file size for writing (must be OODLEX_IO_MAX_ALIGNMENT aligned)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Same as OodleXIOQ_OpenForWriteCreate_Async except that it creates a unique temp name to write to. The temp name starts with nameBase, if given. Providing nameBase is helpful because it lets Oodle put the temp file in the same directory as the final file name, which ensures that the final rename can be done without copying.

Should be used with OodleXIOQ_CloseFileRename_Async.

Writing to a temp name and then renaming over the desired output file only on successful completion is the recommended way to write all files. It means you won't destroy the user's data by failing to successfully overwrite a previously existing good file.
 

OodleXIOQ_OpenForWriteCreate_AsyncOodleX low level async ioOodleXIOQ_CloseFile_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_LogSystemInfo
Navigation
 OodleX_Init_Default
 OodleX_Shutdown
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_LogSystemInfo( );
Discussion
Log some info about the platform
Discussion
This function should be called after OodleX_Init.

It prints some info to the Oodle Log about the Oodle build and your system. This is a helpful thing to include in debug reports sent to RAD.
 

OodleX_Init_DefaultOodleX Startup and ShutdownOodleX_Shutdown

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init
Navigation
 OodleX_Init_GetDefaults_Minimal
 OodleX_Init_Default
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleX_Init( OO_U32 oodle_header_version,
                     const OodleXInitOptions * pOptions );
Discussion
Initialize Oodle
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
pOptions  options for Init; must not be NULL; use OodleX_Init_Default if you don't want to set up options
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

You must call OodleX_Init or OodleX_Init_NoThreads before any other Oodle function that you expect to work.

Pair with OodleX_Shutdown.

For minimal linkage, use OodleX_Init_NoThreads
 

OodleX_Init_GetDefaults_MinimalOodleX Startup and ShutdownOodleX_Init_Default

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFileMode
Navigation
 OodleXError
 OodleXFileOpenFlags
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXFileMode
{
    OodleXFileMode_Invalid = 0,
    OodleXFileMode_Read = 1,
    OodleXFileMode_WriteCreate = 2,
    OodleXFileMode_Write = 2,
    OodleXFileMode_ReadWrite = OodleXFileMode_Read|OodleXFileMode_Write,
    OodleXFileMode_Force32 = 0x40000000
};
Discussion
FileMode used by OodleFile and such. Not all OodleFile types support OodleXFileMode_ReadWrite
Enumerants
OodleXFileMode_Invalid  file mode not set
OodleXFileMode_Read  open existing, shared
OodleXFileMode_WriteCreate  open new (create/trunc), exclusive
OodleXFileMode_Write  alias for OodleXFileMode_WriteCreate
OodleXFileMode_ReadWrite  open existing or create if new, exclusive, read/write
OodleXFileMode_Force32 

 
OodleXErrorOodleX low level async ioOodleXFileOpenFlags

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFreeSized
Navigation
 OodleXFree
 OodleXMallocBigAlignment
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXFreeSized( void * ptr,
                      OO_SINTa bytes );
Discussion
Free a pointer allocated by OodleXMalloc or OodleXMallocAligned
Parameters
ptr  the pointer to free; allocated by OodleXMalloc or OodleXMallocAligned (must not be NULL)
bytes  the size of the allocation as originally requested
Discussion

Providing the size of the malloc allows much faster freeing Size must match the allocated size! Uses the current OodleXMallocVTable ; this is an error if ptr was allocated from a different VTable.
 

OodleXFreeOodleX Memory AllocatorsOodleXMallocBigAlignment

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleCountdown_Alloc
Navigation
 OodleXHandleEvent_SetError
 OodleXHandleCountdown_Decrement
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXHandleCountdown_Alloc( OO_S32 initialCount,
                                          OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ) );
Discussion
Allocate an OodleXHandle to a simple data-less coutdown
Parameters
initialCount  inital count; becomes done when count reaches 0
autoDelete  (optional) set the OodleXHandleAutoDelete of the handle
Return Value
return  the handle
Discussion

initialCount should be greater than 0.

A Countdown is a simple handle which you can use to wait for completion of many tasks. Use OodleXHandleCountdown_Decrement to decrement it. When it reaches 0 it becomes Done, which means it satisfies an OodleX_Wait.

(A countdown is the same thing as a single-use Semaphore with an initial negative count)

If autoDelete is OodleXHandleAutoDelete_Yes , the Countdown handle is deleted when count reaches zero. (a deleted handle also satisfies OodleX_Wait).
 

OodleXHandleEvent_SetErrorOodleX async handle operationsOodleXHandleCountdown_Decrement

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle Network compression
Navigation
 Oodle2 Network API Documentation
 Network plugins
 Welcome to Oodle
 Change Log
  •  OodleAPI_OodleNetwork1
    •  About OodleNetwork1
    •  Defines
      •  OODLENETWORK1_MAX_DICTIONARY_SIZE
      •  OODLENETWORK1_HASH_BITS_DEFAULT
      •  OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
    •  Structures
      •  OodleNetwork1_Shared
      •  OodleNetwork1TCP_State
      •  OodleNetwork1UDP_State
      •  OodleNetwork1UDP_StateCompacted
    •  Functions
      •  OodleNetwork1_Shared_Size
      •  OodleNetwork1TCP_State_Size
      •  OodleNetwork1_CompressedBufferSizeNeeded
      •  OodleNetwork1_Shared_SetWindow
      •  OodleNetwork1TCP_State_Reset
      •  OodleNetwork1TCP_State_InitAsCopy
      •  OodleNetwork1TCP_Train
      •  OodleNetwork1TCP_Encode
      •  OodleNetwork1TCP_Decode
      •  OodleNetwork1UDP_Train
      •  OodleNetwork1UDP_State_Size
      •  OodleNetwork1UDP_Encode
      •  OodleNetwork1UDP_Decode
      •  OodleNetwork1UDP_StateCompacted_MaxSize
      •  OodleNetwork1UDP_State_Compact_ForVersion
      •  OodleNetwork1UDP_State_Compact
      •  OodleNetwork1UDP_State_Uncompact_ForVersion
      •  OodleNetwork1UDP_State_Uncompact
      •  OodleNetwork1_SelectDictionarySupported
      •  OodleNetwork1_SelectDictionaryFromPackets
      •  OodleNetwork1_SelectDictionaryFromPackets_Trials
  •  About Oodle Network Compression
    •  Capturing Training data for OodleNetwork
    •  Forming Packets for Maximum Compression

 
Oodle2 Network API DocumentationOodle2 Network API DocumentationOodleAPI_OodleNetwork1

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
Navigation
 OodleXIOQ_CopyFile_Async
 OodleXIOQ_WriteWholeFile_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXIOQ_ReadMallocWholeFile_AsyncAndWait( const char * vfsName,
                                                   OO_S64 * pSize,
                                                   OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ) );
Discussion
See OodleXIOQ_ReadMallocWholeFile_Async
 
OodleXIOQ_CopyFile_AsyncOodleX low level async ioOodleXIOQ_WriteWholeFile_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleConfigValues
Navigation
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleConfigValues
{
    OO_S32 m_OodleLZ_LW_LRM_step;
    OO_S32 m_OodleLZ_LW_LRM_hashLength;
    OO_S32 m_OodleLZ_LW_LRM_jumpbits;
    OO_S32 m_OodleLZ_Decoder_Max_Stack_Size;
    OO_S32 m_OodleLZ_Small_Buffer_LZ_Fallback_Size_Unused;
    OO_S32 m_OodleLZ_BackwardsCompatible_MajorVersion;
    OO_U32 m_oodle_header_version;
};
Discussion
OodleConfigValues
Members
m_OodleLZ_LW_LRM_step  LZHLW LRM : bytes between LRM entries
m_OodleLZ_LW_LRM_hashLength  LZHLW LRM : bytes hashed for each LRM entries
m_OodleLZ_LW_LRM_jumpbits  LZHLW LRM : bits of hash used for jump table
m_OodleLZ_Decoder_Max_Stack_Size  if OodleLZ_Decompress needs to allocator a Decoder object, and it's smaller than this size, it's put on the stack instead of the heap
m_OodleLZ_Small_Buffer_LZ_Fallback_Size_Unused  deprecated
m_OodleLZ_BackwardsCompatible_MajorVersion  if you need to encode streams that can be read with an older version of Oodle, set this to the Oodle2 MAJOR version number that you need compatibility with. eg to be compatible with oodle 2.7.3 you would put 7 here
m_oodle_header_version  = OODLE_HEADER_VERSION
Discussion
Struct of user-settable low level config values. See Oodle_SetConfigValues.

May have different defaults per platform.
 

Oodle_UsageWarningsCore BaseOodle_GetConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMallocFailedHandler
Navigation
 Welcome to Oodle
 Change Log
// Function typedef:
OO_BOOL( OODLE_CALLBACK OodleXMallocFailedHandler )( OO_SINTa bytes );
Discussion
OodleXMallocFailedHandler is called when a malloc fails Return true to retry.
 
OodleXMalloc_GetVTable_OSOodleX Memory AllocatorsOodleX async handle operations

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFree
Navigation
 OodleXMallocAligned
 OodleXFreeSized
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXFree( void * ptr );
Discussion
free a pointer allocated by OodleXMalloc or OodleXMallocAligned
Parameters
ptr  the pointer to free (must not be NULL)
Discussion

Uses the current OodleXMallocVTable ; this is an error if ptr was allocated from a different VTable. Prefer OodleXFreeSized whenever possible.
 

OodleXMallocAlignedOodleX Memory AllocatorsOodleXFreeSized

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: What are the Oodle deprecated compressors ?
Navigation
 FAQ: How much memory do the Oodle compressors use ?
 Frequently Asked Questions
 FAQ: Do new Oodle versions break data compatibility ?
 Welcome to Oodle
 Change Log

The new Oodle compressor family consists of Kraken, Leviathan, Mermaid, Selkie, and Hydra.

New Oodle users should only use the new sea monster family of compressors.

LZB16 is also supported but not recommended.

As of Oodle 2.9.0 the older compressors (hidden in OODLE_ALLOW_DEPRECATED_COMPRESSORS) are no longer supported. They cannot be encoded or decoded.

If you have old data from Oodle 2.8.x or earlier, you can continue to use the old Oodle lib.
 

FAQ: How much memory do the Oodle compressors use ?Frequently Asked QuestionsFAQ: Do new Oodle versions break data compatibility ?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetName
Navigation
 OodleXIOQ_GetLastPendingOpOnFile
 OodleXIOQ_GetLastError
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_GetName( OodleXIOQFile file,
                           char * pInto,
                           OO_S32 intoSize );
Discussion
Get the file name (OS name)
Parameters
file  the IOQFile to query
pInto  filled with the file's OS name (UTF8)
intoSize  number of bytes Oodle can write to pInto
Return Value
return  true if successful
Discussion

Copies the OS name (UTF8) into pInto. This may not be the same as the name used when opening the file, if that was a VFS name.
 

OodleXIOQ_GetLastPendingOpOnFileOodleX low level async ioOodleXIOQ_GetLastError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz_noallocs : Example demonstrating Oodle compression with no allocations
Navigation
 example_lz_chart : Example that makes a chart of OodleLZ options
 Examples
 example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz_noallocs

Very simple example of OodleLZ memory -> memory compression & decompression.

Shows how to use Oodle without any allocations done by Oodle. All memory needed is passed in by the client.

Uses stdio for file IO to load an input file.

See example_lz : Example demonstrating LZ compression and decompression for more advanced OodleLZ usage.

example_lz_noallocs only uses Oodle Core, no Oodle Ext

include oodle2.h and not oodle2x.h

If you want to use Oodle with no allocations, you cannot use OodleX. OodleX installs its own allocator into Oodle Core. Do not use OodlePlugins_SetAllocators with OodleX.

See FAQ: How do I use Oodle with no allocator? and OodleCore_Plugins_SetAllocators


#include "../include/oodle2.h"

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz_noallocs
#endif

#include "read_whole_file.h"


void * OODLE_CALLBACK example_noallocs_MallocAligned_Logging(OO_SINTa size,OO_S32 alignment)
{
    // malloc should not be called, log an error :
    printf("ERROR : example_noallocs_MallocAligned_Logging called (size %d)\n",(int)size);

    // use default Oodle MallocAligned as an easy way to get a cross-platform aligned malloc :
    return OodleCore_Plugin_MallocAligned_Default(size,alignment);
}

void OODLE_CALLBACK example_noallocs_Free_Logging(void * ptr)
{
    // malloc should not be called, log an error :
    printf("ERROR : example_noallocs_Free_Logging called.\n");
    
    OodleCore_Plugin_Free_Default(ptr);
}


extern "C" int main(int argc,char *argv[])
{
No initialization is needed for Oodle2 Core

we let Oodle Core use the default system plugins (the C stdlib) To change them, use Core plugins


    // optional check to make sure header matches lib :
    if ( ! Oodle_CheckVersion(OODLE_HEADER_VERSION) )
    {
        fprintf(stderr,"Oodle header version mismatch\n");
        return 10;
    }

Install our own allocator plugins that log an error if called.

These should never be called. You could also disable them : OodleCore_Plugins_SetAllocators(NULL,NULL);

but that is not recommended because it will cause a hard failure if Oodle ever needs memory.

    OodleCore_Plugins_SetAllocators(example_noallocs_MallocAligned_Logging,example_noallocs_Free_Logging);

    // get args :
    const char * in_name;
    if ( argc < 2 )
    {
        in_name = "r:\\testsets\\lztestset\\lzt02";
    }
    else
    {
        in_name = argv[1];
    }
    
read input file using stdio
    OO_SINTa length;
    void * buf = read_whole_file(in_name,&length);
    if ( ! buf )
    {
        fprintf(stderr,"couldn't open : %s\n",in_name);
        return 10;
    }
    

Run OodleLZ_Compress from memory (buf) to memory (compbuf)

Use the OodleLZ_Compressor_Kraken compressor. Kraken is an amazing balance of good compression and fast decode speed. It should generally be your first choice.

Use OodleLZ_CompressionLevel_Normal level of effort in the encoder. Normal is a balance of encode speed and compression ratio. Different levels trade off faster or slower encoding for compressed size. See OodleLZ_CompressionLevel.

See About OodleLZ for information on selection of the compression options.

This call is synchronous and not threaded; see example_lz : Example demonstrating LZ compression and decompression for an example using the async compression APIs.


    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Normal;
    //OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Optimal2; // optimals are OODLELZ_SCRATCH_MEM_NO_BOUND

    // allocate memory big enough for compressed data :
    void * compbuf = malloc( OodleLZ_GetCompressedBufferSizeNeeded(compressor,length) + sizeof(length) );
    if ( compbuf == NULL )
        return 10;

    // allocate memory for encoder scratch :
    OO_SINTa enc_scratch_size = OodleLZ_GetCompressScratchMemBound(compressor,level,length,NULL);
    if ( enc_scratch_size == OODLELZ_SCRATCH_MEM_NO_BOUND )
    {
        // scratch cannot be bounded for this choice of compressor/level
        // the allocator may be used!
        // go ahead and give it 4 MB of scratch
        enc_scratch_size = 4*1024*1024;
    }
    
    void * enc_scratch = malloc(enc_scratch_size);
    if ( enc_scratch == NULL )
        return 10;  
        
    char * compptr = (char *)compbuf;
    memcpy(compptr,&length,sizeof(length));
    compptr += sizeof(length);

    // compress :
    OO_SINTa complen = OodleLZ_Compress(compressor,buf,length,compptr,level,NULL,NULL,NULL,enc_scratch,enc_scratch_size);
    compptr += complen;

    // log about it :
    //  full compressed size also includes the header
    printf("%s compressed %d -> %d (+%d)\n",in_name,(int)length,(int)complen,(int)sizeof(length));

    // can free enc_scratch now
    // enc_scratch can be reused for further compression
    //  but must be used by only one thread at a time
    free(enc_scratch); enc_scratch = NULL;

Run OodleLZ_Decompress from memory (compbuf) to memory (decbuf)

Note that you must provide the exact decompressed size. OodleLZ data is headerless; store the size in your own header.


    OO_SINTa declength;
    compptr = (char *)compbuf;
    memcpy(&declength,compptr,sizeof(declength));
    compptr += sizeof(declength);
    assert( length == declength );

    // malloc for decompressed buffer  :
    void * decbuf = malloc( declength );
    if ( decbuf == NULL )
        return 10;      

    OO_SINTa decoder_mem_size = OodleLZDecoder_MemorySizeNeeded(compressor,declength);
    void * decoder_mem = malloc(decoder_mem_size);
    if ( decoder_mem == NULL )
        return 10;      

    // do the decompress :
    OO_SINTa decompress_return = OodleLZ_Decompress(compptr,complen,decbuf,declength,
            OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_Yes,OodleLZ_Verbosity_None,NULL,0,NULL,NULL,
            decoder_mem,decoder_mem_size,
            OodleLZ_Decode_Unthreaded);
    
    // check it was successful :
    assert( decompress_return == length );
    assert( memcmp(buf,decbuf,length) == 0 );

    if ( decompress_return != length )
        return 10;

    printf("decompessed successfully.\n");
        
And finish up. No shutdown is needed for Oodle2 Core.

    // free all the memory :
    free(buf);
    free(compbuf);
    free(decbuf);
    free(decoder_mem);

    return 0;
}

 
example_lz_chart : Example that makes a chart of OodleLZ optionsExamplesexample_lz_overlap : Example demonstrating parallel overlap with OodleLZ

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_Printf
Navigation
 t_fp_OodleNet_Plugin_WaitJob
 t_fp_OodleNet_Plugin_DisplayAssertion
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleNet_Plugin_Printf )( int verboseLevel,
                const char * file,
                int line,
                const char * fmt,
                . . . );
Discussion
Function pointer to Oodle Core printf
Parameters
verboseLevel  verbosity of the message; 0-2 ; lower = more important
file  C file that sent the message
line  C line that sent the message
fmt  vararg printf format string
Discussion

The logging function installed here must parse varargs like printf.

verboseLevel may be used to omit verbose messages.
 

t_fp_OodleNet_Plugin_WaitJobNetwork pluginst_fp_OodleNet_Plugin_DisplayAssertion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetCompressedBufferSizeNeeded
Navigation
 OodleLZ_GetCompressScratchMemBoundEx
 OodleLZ_GetDecodeBufferSize
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetCompressedBufferSizeNeeded( OodleLZ_Compressor compressor,
                                                OO_SINTa rawSize );
Discussion
Return the size you must malloc the compressed buffer
Parameters
compressor  compressor used; OodleLZ_Compressor_Invalid to make it enough for any compressor
rawSize  uncompressed size you will compress into this buffer
Discussion

The compBuf passed to OodleLZ_Compress must be allocated at least this big.

note this is actually larger than the maximum size of a compressed stream, it includes overrun padding.


 

OodleLZ_GetCompressScratchMemBoundExOodleAPI_LZ_CompressorsOodleLZ_GetDecodeBufferSize

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetStatus
Navigation
 OodleX_Wait
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleX_GetStatus( OodleXHandle h,
                               OodleXHandleDeleteIfDone deleteIfDone OODEFAULT( OodleXHandleDeleteIfDone_No ) );
Discussion
Get the Status of an async handle
Parameters
h  OodleXHandle weak reference
deleteIfDone  if OodleXHandleDeleteIfDone_Yes and handle is not pending, it is deleted
Return Value
return  handle status
Discussion

This function does not block. Returns OodleXStatus_Invalid if the handle was already deleted or does not exist. Test status for done by checking >= OodleXStatus_Done, because that also includes Error.
 

OodleXHandleDeleteIfDoneOodleX async handle operationsOodleX_Wait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Shutdown
Navigation
 OodleX_LogSystemInfo
 OodleX_Init_NoThreads
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_Shutdown( const char * threadProfileLogName OODEFAULT( NULL ),
                      OodleX_Shutdown_LogLeaks logLeaks OODEFAULT( OodleX_Shutdown_LogLeaks_Yes ),
                      OO_U64 allocStartCounter OODEFAULT( 0 ),
                      OodleX_Shutdown_DebugBreakOnLeaks debugBreakOnLeaks OODEFAULT( OodleX_Shutdown_DebugBreakOnLeaks_No ) );
Discussion
Shut down Oodle at app exit time.
Parameters
threadProfileLogName  (optional) if not NULL, and the ThreadProfiler is enabled, writes the threadprofiler output to this file name
logLeaks  (optional) if true and the LeakTracker is enabled, logs any leaks or memory or handles
allocStartCounter  (optional) initial counter for the LeakTrack log
debugBreakOnLeaks  (optional) if there are any leaks, do a debug break
Discussion

Pair with OodleX_Init. No Oodle functions should be called after Shutdown.

Call Shutdown from the same thread that called Init.

Do not shutdown Oodle then init again. Only call Init and Shutdown once per run.
 

OodleX_LogSystemInfoOodleX Startup and ShutdownOodleX_Init_NoThreads

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_Reset
Navigation
 OodleLZDecoder_Destroy
 OodleLZDecoder_DecodeSome
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZDecoder_Reset( OodleLZDecoder * decoder,
                              OO_SINTa decPos,
                              OO_SINTa decLen OODEFAULT( 0 ) );
Discussion
Reset an OodleLZDecoder to restart at given pos
Parameters
decoder  the OodleLZDecoder, made by OodleLZDecoder_Create
decPos  position to reset to; must be a multiple of OODLELZ_BLOCK_LEN
decLen  (optional) if not zero, change the length of the data we expect to decode
Return Value
return  true for success
Discussion

If you are seeking in a packed stream, you must seek to a seek chunk reset point, as was made at compress time.

That is, OodleLZ_CompressOptions:seekChunkReset must have been true, and decPos must be a multiple of OodleLZ_CompressOptions:seekChunkLen that was used at compress time.

You can use OodleLZ_GetChunkCompressor to verify that you are at a valid independent chunk start point.


 

OodleLZDecoder_DestroyOodleAPI_LZ_CompressorsOodleLZDecoder_DecodeSome

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_Flush
Navigation
 OodleXLog_SetVerboseLevel
 OodleXLog_PrintfError
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXLog_Flush( );
Discussion
Flush output to the log
Discussion
Close & reopen the log file to ensure data is on disk. Useful when trying to debug a crash.

OodleXLog_Flush also prints queued messaged from threads, namely the IOQ Log which you turn on with OodleXInitOptions:m_OodleInit_IOQ_Log. If you are using the IOQ log, call OodleXLog_Flush once per frame or so when you want to see its messages.

You can also use OODLEXLOG_AUTOFLUSH_THREADLOG in the log State, but that only has affect when you do OodleLog Printfs.
 

OodleXLog_SetVerboseLevelOodleX Debug aidsOodleXLog_PrintfError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
Navigation
 OodleXIOQ_OpenAndReadMallocWholeFile_Async
 OodleXIOQ_OpenWriteWholeFileClose_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async( const char * name,
                                                                 OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                                 const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                                 OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                                 OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                                 const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                                 OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to open a file, allocate a buffer for a whole file and read it
Parameters
name  name of the file to open (VFS, UTF-8)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
Return Value
return  handle to the RMWF op; use OodleXIOQ_ReadMallocWholeFile_GetResult
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

Performs OodleXIOQ_OpenForRead_Async and OodleXIOQ_ReadMallocWholeFile_Async and OodleXIOQ_CloseFile_Async.

The OodleXHandle returned is to the RMWF operation; use OodleXIOQ_ReadMallocWholeFile_GetResult.
 

OodleXIOQ_OpenAndReadMallocWholeFile_AsyncOodleX low level async ioOodleXIOQ_OpenWriteWholeFileClose_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleCore_Plugins_SetJobSystem
Navigation
 OodleCore_Plugins_SetAllocators
 OodleCore_Plugins_SetJobSystemAndCount
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleCore_Plugins_SetJobSystem( t_fp_OodleCore_Plugin_RunJob * fp_RunJob,
                                     t_fp_OodleCore_Plugin_WaitJob * fp_WaitJob );
Discussion
DEPRECATED use OodleCore_Plugins_SetJobSystemAndCount instead
Discussion
See OodleCore_Plugins_SetJobSystemAndCount
 
OodleCore_Plugins_SetAllocatorsCore pluginsOodleCore_Plugins_SetJobSystemAndCount

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_ThreadFunc
Navigation
 OodleX_Semaphore
 Welcome to Oodle
 Change Log
// Function typedef:
OO_U32( OODLE_CALLBACK OodleX_ThreadFunc )( void * userdata );
Discussion
User-provided callback for threads
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX_SemaphoreOodleX threading utilOodleX low level async io

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetFirstChunkCompressor
Navigation
 OodleLZ_GetAllChunksCompressor
 OodleLZ_GetChunkCompressor
 Welcome to Oodle
 Change Log
// Function prototype:
OodleLZ_Compressor OodleLZ_GetFirstChunkCompressor( const void * compChunkPtr,
                                                    OO_SINTa compBufAvail,
                                                    OO_BOOL * pIndependent );
Discussion
ask who compressed this chunk
Parameters
compChunkPtr  pointer to compressed data; must be the start of compressed buffer, or a step of OODLELZ_BLOCK_LEN raw bytes
compBufAvail  number of bytes at compChunkPtr available to read
pIndependent  (optional) filled with a bool for whether this chunk is independent of predecessors
Return Value
return  the OodleLZ_Compressor used to encode this chunk
Discussion

note this is only for this chunk - later chunks may have different compressors (eg. with Hydra) if you compressed all chunks the same it's up to you to store that info in your header

Use OodleLZ_GetAllChunksCompressor for data that might be mixed compressors.

This replaces the deprecated function OodleLZ_GetChunkCompressor

returns OodleLZ_Compressor_Invalid if compBufAvail is too small or the chunk is corrupt
 

OodleLZ_GetAllChunksCompressorOodleAPI_LZ_CompressorsOodleLZ_GetChunkCompressor

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetNumSeekChunks
Navigation
 OodleLZ_MakeSeekChunkLen
 OodleLZ_GetSeekTableMemorySizeNeeded
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZ_GetNumSeekChunks( OO_S64 rawLen,
                                 OO_S32 seekChunkLen );
Discussion
Compute the number of seek chunks
Parameters
rawLen  total length of uncompressed data
seekChunkLen  the length of a seek chunk (eg from OodleLZ_MakeSeekChunkLen)
Return Value
return  the number of seek chunks
Discussion

returns (rawLen+seekChunkLen-1)/seekChunkLen
 

OodleLZ_MakeSeekChunkLenOodleAPI_LZ_CompressorsOodleLZ_GetSeekTableMemorySizeNeeded

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFileOpsVTable
Navigation
 OodleXFileInfo
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleXFileOpsVTable
{
    OodleXOSFile (OODLE_CALLBACK *sync_open)(const OodleXFileOpsVTable * vtable,const char * name,OodleXFileMode mode,OodleXFileOpenFlags flags, OO_S64 reserveSize, OO_S32 * pAlignmentRequired );
    OO_BOOL (OODLE_CALLBACK *sync_close)(const OodleXFileOpsVTable * vtable,OodleXOSFile handle,const char * name, OO_S64 truncateSize );
    OO_BOOL (OODLE_CALLBACK *sync_get_file_info)(const OodleXFileOpsVTable * vtable,OodleXOSFile handle,OodleXFileInfo * info);
    OO_BOOL (OODLE_CALLBACK *sync_read)(const OodleXFileOpsVTable * vtable, OodleXOSFile handle, OO_S64 pos, OO_S32 size, void * memory, OO_S32 * pGotSize);
    OO_BOOL (OODLE_CALLBACK *sync_write)(const OodleXFileOpsVTable * vtable,OodleXOSFile handle, OO_S64 pos, OO_S32 size, const void * memory);
    OO_BOOL (OODLE_CALLBACK *sync_set_file_size)(const OodleXFileOpsVTable * vtable,OodleXOSFile handle, OO_S64 size);
    OO_U32      (OODLE_CALLBACK *get_last_error)(const OodleXFileOpsVTable * vtable,OodleXOSFile handle);
    OodleXError (OODLE_CALLBACK *get_error_enum)(const OodleXFileOpsVTable * vtable,OO_U32 err);
    void    (OODLE_CALLBACK *get_error_string)(const OodleXFileOpsVTable * vtable,OO_U32 err,char * pInto,int intoSize);
    OO_BOOL (OODLE_CALLBACK *delete_file)(const OodleXFileOpsVTable * vtable,const char * name);
    OO_BOOL (OODLE_CALLBACK *rename_file)(const OodleXFileOpsVTable * vtable,const char * fm,const char * to,OO_BOOL overwrite);
    OodleXOSFileListing (OODLE_CALLBACK *listdir_open)(const OodleXFileOpsVTable * vtable,const char * dir);
    void    (OODLE_CALLBACK *listdir_close)(const OodleXFileOpsVTable * vtable,OodleXOSFileListing handle);
    OO_BOOL (OODLE_CALLBACK *listdir_next)(const OodleXFileOpsVTable * vtable,OodleXOSFileListing handle,const char* path,char * name,int nameSize,OodleXFileInfo * info);
    OO_BOOL (OODLE_CALLBACK *get_filename_info)(const OodleXFileOpsVTable * vtable,const char * name,OodleXFileInfo * info);
    OO_BOOL (OODLE_CALLBACK *set_filename_info)(const OodleXFileOpsVTable * vtable,const char * name,OO_U32 flags,OO_U64 modtime);
    OO_BOOL (OODLE_CALLBACK *force_writeable)(const OodleXFileOpsVTable * vtable,const char * name);
    OO_BOOL (OODLE_CALLBACK *make_dir)(const OodleXFileOpsVTable * vtable,const char * name);
    void *  fileops_data;
};
Discussion
VTable struct of lowest level file ops
Members
sync_open  open file in desired mode; reserveSize is for WriteCreate only
sync_close  close a file; must provide name if truncateSize is not -1 ; truncateSize only applies for files open for Write ; truncate may happen after closing, non-atomically
sync_get_file_info  fill out info
sync_read  read from a file at pos; pos, size & memory must all be aligned; fill out pGotSize; *pGotSize must == size if you return true unless you are at EOF
sync_write  write to a file at pos; pos, size & memory must all be aligned; must write full size if you return true
sync_set_file_size  set file size; size must be aligned! to set unaligned size, use truncateSize on close
get_last_error  returns an OS error code
get_error_enum  maps an OS error code to an OodleXError enum
get_error_string  takes OS error code, fills pInto (const OodleXFileOpsVTable * vtable,UTF8)
delete_file  delete by name
rename_file  rename by name ; overwrite existing only if passed true
listdir_open  returns 0 if dir does not exist
listdir_close  pair with listdir_open
listdir_next  listdir_next returns true if info is filled out ; name is UTF8
get_filename_info  get info by name
set_filename_info  set OodleXFileInfo:flags and/or modtime (const OodleXFileOpsVTable * vtable,use OODLEX_FILEINFO_FLAG_INVALID and OODLEX_FILEINFO_MODTIME_INVALID to not change)
make_dir  make a dir
fileops_data  set by the creator the vtable and used however you like
Discussion
IOQ acts through a VTable , which allows you to plug in your own functions to bottom level IO


 

OodleXFileOpenFlagsOodleX low level async ioOodleXFileInfo

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle Network Compression
Navigation
 OodleAPI_OodleNetwork1
 Oodle Network compression
 Welcome to Oodle
 Change Log

Oodle Network Compression provides realtime compression & decompression of game packets.

The primary compressor which you probably want is OodleNetwork1. See About OodleNetwork1 .

If you are evaluating Oodle Network, the main thing you need to do is to capture a large data set for testing and training. See Capturing Training data for OodleNetwork

Note that network packet compression is separate from compression of static packages. eg. for sending game updates or content, you would compress the data offline and just store it compressed on your server. For that, you should use the normal Oodle LZ compressors. See About OodleLZ.

  •  Capturing Training data for OodleNetwork
  •  Forming Packets for Maximum Compression

  • About OodleNetwork1
  • OodleAPI_OodleNetwork1

 
OodleNetwork1UDP_StateCompactedOodle Network compressionCapturing Training data for OodleNetwork

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: OodleLZ_Decompress is failing how do I diagnose it?
Navigation
 Frequently Asked Questions
 FAQ: How do I use Oodle with no allocator?
 Welcome to Oodle
 Change Log

If you are getting OODLELZ_FAILED and having trouble isolating it, here are some suggestions.

For OodleLZ_Decompress to succeed, the compressed bytes must be correct, and the arguments you pass in (compBufSize and rawLen) must be correct. The raw size that the data decompresses to is not stored in the compressed data, it's up to you to pass that in, and if that's wrong it will show up as OODLELZ_FAILED.

The most common cause of this is that the compressed bytes are corrupted or weren't filled. This can happen if the IO to fill them actually failed, or some mismatch about the number of compressed bytes to load. It can happen if an async IO was used but it isn't actually done yet at the time of the Decompress call (which would be a race).

There are a couple of things you can do to try to track it down :

Checksum the compressed data and verify it is valid before decompressing. You can either do your own checksums or use the system in Oodle.

To use the system in Oodle, you must encode the data with OodleLZ_CompressOptions:sendQuantumCRCs enabled (they are off by default), then in the OodleLZ_Decompress pass OodleLZ_CheckCRC_Yes.

Oodle's CRC's are on the compressed data and are checked before the actual decompression. This will still give you an OODLELZ_FAILED but will log about crc failure.

To get the debug logs and perhaps more details :

Use the Debug build of Oodle. Verbose logs are not compiled into the release build.

Enable full log verbosity :

OodlePlugin_Printf_Default does not show verbose logs. Install OodlePlugin_Printf_Verbose with OodleCore_Plugins_SetPrintf.

(If using OodleX, doesn't do that, instead use OodleXLog_SetVerboseLevel.)

Another thing you can do :

When you get OODLELZ_FAILED from OodleLZ_Decompress, add a branch to your code that detects that case and writes out the compressed buffer that was passed in, as well as all the arguments to the OodleLZ_Decompress function. Also write out where you expected that compressed buffer to come from in a file.

Now go to the file that should contain that compressed data and diff it against the compressed data you logged out; do they match? This can help to detect IO races and memory corruption.

Another thing that can be helpful to check for races :

After seeing an OODLELZ_FAILED return, pause and then repeat the same decompression again. If it now succeeds, that could indicate a race.

OodleLZ is fully thread safe and does not access any globals, however the compressed buffer, the output decompressed buffer, and the decoder memory passed in should only be used by the current thread.

Note that races are not just possible in the compressed buffer that OodleLZ_Decompress reads; the output uncompressed buffer is also read-write, so another thread writing in there could cause an OODLELZ_FAILED, as could another thread using the passed in decoder scratch memory.

If you've done all these things and are still lost, contact me. I'll need you to send me the results of what you've found in the above steps, as well as the log of all the function arguments & the compressed buffer contents passed to OodleLZ_Decompress so I can reproduce the function call.
 

Frequently Asked QuestionsFrequently Asked QuestionsFAQ: How do I use Oodle with no allocator?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Profile
Navigation
 OodleLZ_CheckCRC
 OodleDecompressCallbackRet
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_Profile
{
    OodleLZ_Profile_Main = 0,
    OodleLZ_Profile_Reduced = 1,
    OodleLZ_Profile_Force32 = 0x40000000
};
Discussion
Decode profile to target
Enumerants
OodleLZ_Profile_Main  Main profile (all current features allowed)
OodleLZ_Profile_Reduced  Reduced profile (Kraken only, limited feature set)
OodleLZ_Profile_Force32 

 
OodleLZ_CheckCRCOodleAPI_LZ_CompressorsOodleDecompressCallbackRet

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenForWriteCreate_Async
Navigation
 OodleXIOQ_OpenAndRead_Async
 OodleXIOQ_OpenForWriteTempName_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenForWriteCreate_Async( OodleXIOQFile * pFile,
                                                 const char * name,
                                                 OO_S64 initialFileSize OODEFAULT( OODLEX_FILE_OPEN_NO_RESERVE_SIZE ),
                                                 OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                 const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                 OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                 OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                 const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                 OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start opening a file for write
Parameters
pFile  filled with a handle to the file which will be opened
name  name of the file to open (VFS, UTF-8)
initialFileSize  (optional) pre-allocate file size for writing (must be OODLEX_IO_MAX_ALIGNMENT aligned)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

See many shared aspects of OodleXIOQ_OpenForRead_Async .

OpenForWrite opens files as create/truncate with exclusive access.

initialFileSize performs an initial pre-allocation of file space, same as OodleXIOQ_ReserveFileSizeForWrite_Async. Pre-allocated file space has undefined (garbage) contents. Writes are faster to pre-allocated space.

WARNING : WriteCreate will overwrite (stomp) existing files by default. If you don't want that, pass OodleXFileOpenFlags_WriteCreateDontStomp in fileOpenFlags.
 

OodleXIOQ_OpenAndRead_AsyncOodleX low level async ioOodleXIOQ_OpenForWriteTempName_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_VerboseLevel
Navigation
 OodleXLog_StateFlags
 OodleXLogCallbackRetRet
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXLog_VerboseLevel
{
    OodleXLog_Verbose_None = -1,
    OodleXLog_Verbose_Minimal = 0,
    OodleXLog_Verbose_Some = 1,
    OodleXLog_Verbose_Lots = 2,
    OodleXLog_Verbose_Force32 = 0x40000000
};
Discussion
Standard verbosity levels for use with OodleXLog_SetVerboseLevel
Enumerants
OodleXLog_Verbose_None  log nothing
OodleXLog_Verbose_Minimal  log only very important messages, such as errors
OodleXLog_Verbose_Some  default setting during development
OodleXLog_Verbose_Lots  log lots; may be slow (note: these are compiled out in release builds)
OodleXLog_Verbose_Force32 

 
OodleXLog_StateFlagsOodleX Debug aidsOodleXLogCallbackRetRet

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Index of Abouts
Navigation
 About Oodle
 About Oodle ozip
 Welcome to Oodle
 Change Log

  • About Oodle Ext

  • About Oodle Network Compression

  • About OodleLZ

  • About OodleNetwork1

  • About OodleXHandle

  • About OodleIOQ

  • About OodleXMalloc

  • About Oodle on Platforms

  • How to build and use the Oodle examples

  • About Oodle Job Threading Plugins

  • About Oodle ozip

 
About OodleAbout OodleAbout Oodle ozip

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_ReadAndDecompress_Stream_Async
Navigation
 OodleXLZ_Compress_AsyncAndWait
 OodleXDecompressCallback_WriteFile_Data_Init
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_ReadAndDecompress_Stream_Async( OO_U32 asyncSelect,
                                                      const void * packedDataPtr,
                                                      OO_SINTa packedLen,
                                                      void * rawPtr,
                                                      OO_SINTa rawChunkLen,
                                                      OodleLZ_FuzzSafe fuzzSafe,
                                                      OodleLZ_CheckCRC checkCRC,
                                                      OodleLZ_Verbosity verbosity,
                                                      OodleDecompressCallback * pcb,
                                                      void * pcbData,
                                                      OodleXIOQFile readFile,
                                                      void * readBuf,
                                                      OO_S64 readStartPos,
                                                      OodleXHandle readPending,
                                                      OO_SINTa alreadyReadSize,
                                                      OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                      const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                      OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an async op to incrementally stream in data and decompress
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run (but Wide is ignored, this func is always narrow)
packedDataPtr  pointer to start of compressed data
packedLen  length of compressed data
rawPtr  pointer to memory to decompress into
rawChunkLen  lenght of raw data to decompress
checkCRC  if OodleLZ_CheckCRC_Yes, the decompressor checks the crc to ensure data integrity
verbosity  if not OodleLZ_Verbosity_None, will log some information
pcb  OodleDecompressCallback called during decompression (NULL for none)
pcbData  user data passed to pcb
readFile  IOQ file to read compressed data from (0 for none)
readBuf  pointer to memory where the reads from readFile should go (must be IO aligned)
readStartPos  file position where readBuf starts (must be IO aligned)
readPending  handle to previously fired read on the IOQ file
alreadyReadSize  the number of bytes of readBuf that are already read (not the number in packedDataPtr)
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation
Discussion

OodleLZ_Async_Decompress_ReadStream : coroutine streaming LZ decoder ; does incremental file reads (optionally - readFile can be zero) ; calls back decode progress so you can do incremental writes (or whatever) ; does not need a seek table (reads raw LZ data).

OodleLZ_Async_Decompress_ReadStream is not "wide" (only one thread is used). It can be used to overlap IO with decompression, but doesn't multi-thread decompression, even if the LZ data has seek chunks.

OodleXLZ_ReadAndDecompress_Stream_Async reads raw LZ data.

OodleXLZ_ReadAndDecompress_Stream_Async is mainly used when you want small granularity incremental callbacks; if you only need OODLELZ_BLOCK_LEN callbacks, then OodleXLZ_Decompress_Narrow_Async is generally better, and OodleXLZ_ReadAndDecompress_Wide_Async is fastest if you want "Wide" async decompression.

packedDataPtr should be somewhere inside readBuf (if the packed data is at the start of the file, they are equal). That is, (packedDataPtr - readBuf + readStartPos) is the position in the file where compressed data starts. Note that readBuf and readStartPos must be IO aligned, but packedDataPtr does not need to be, so to read compressed data from a non-aligned

If provided, the OodleDecompressCallback is called as quanta of raw data are available. The callback may be called more often than OODLELZ_BLOCK_LEN granularity.

Set OodleDecompressCallback to OodleDecompressCallback_WriteFile to perform a streaming read-compress-write.


 

OodleXLZ_Compress_AsyncAndWaitOodleXAPI_LZ_AsyncOodleXDecompressCallback_WriteFile_Data_Init

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_FindSeekEntry
Navigation
 OodleLZ_CheckSeekTableCRCs
 OodleLZ_GetSeekEntryPackedPos
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZ_FindSeekEntry( OO_S64 rawPos,
                              const OodleLZ_SeekTable * seekTable );
Discussion
Find the seek entry that contains a raw position
Parameters
rawPos  uncompressed position to look for
seekTable  result of OodleLZ_CreateSeekTable
Return Value
return  a seek entry index
Discussion

returns the index of the chunk that contains rawPos
 

OodleLZ_CheckSeekTableCRCsOodleAPI_LZ_CompressorsOodleLZ_GetSeekEntryPackedPos

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_SelectDictionarySupported
Navigation
 OodleNetwork1UDP_State_Uncompact
 OodleNetwork1_SelectDictionaryFromPackets
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1_SelectDictionarySupported( void );
Discussion
Returns whether this version of the library can build new dictionaries.
Discussion
This functionality is only available on host platforms, not embedded devices, game consoles, phones etc.
 
OodleNetwork1UDP_State_UncompactOodleAPI_OodleNetwork1OodleNetwork1_SelectDictionaryFromPackets

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Core LZ compression
Navigation
 Core plugins
 Oodle2 Core API Documentation
 Welcome to Oodle
 Change Log

Oodle2 Core LZ lossless data compression.

  •  About OodleLZ
  •  About OodleLZ ThreadPhased Decode
  •  About OodleLZ Hydra
  •  OodleAPI_LZ_Compressors
    •  Defines
      •  OODLE_ALLOW_DEPRECATED_COMPRESSORS
      •  OODLELZ_LOCALDICTIONARYSIZE_MAX
      •  OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
      •  OODLELZ_BLOCK_LEN
      •  OODLELZ_BLOCK_MAX_COMPLEN
      •  OODLELZ_QUANTUM_LEN
      •  OODLELZ_FAILED
      •  OODLELZ_SCRATCH_MEM_NO_BOUND
    •  Enumerants
      •  OodleLZ_Verbosity
      •  OodleLZ_Compressor
      •  OodleLZ_PackedRawOverlap
      •  OodleLZ_CheckCRC
      •  OodleLZ_Profile
      •  OodleDecompressCallbackRet
      •  OodleLZ_CompressionLevel
      •  OodleLZ_Jobify
      •  OodleLZ_Decode_ThreadPhase
      •  OodleLZ_FuzzSafe
      •  OodleLZSeekTable_Flags
      •  OodleLZ_CompressScratchMemBoundType
    •  Structures
      •  OodleLZ_CompressOptions
      •  OodleLZ_DecodeSome_Out
      •  OodleLZDecoder
      •  OodleLZ_SeekTable
    •  Functions
      •  OodleLZ_Compress
      •  OodleLZ_Decompress
      •  OodleLZDecoder_Create
      •  OodleLZDecoder_MemorySizeNeeded
      •  OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
      •  OodleLZDecoder_Destroy
      •  OodleLZDecoder_Reset
      •  OodleLZDecoder_DecodeSome
      •  OodleLZDecoder_MakeValidCircularWindowSize
      •  OodleLZ_MakeSeekChunkLen
      •  OodleLZ_GetNumSeekChunks
      •  OodleLZ_GetSeekTableMemorySizeNeeded
      •  OodleLZ_FillSeekTable
      •  OodleLZ_CreateSeekTable
      •  OodleLZ_FreeSeekTable
      •  OodleLZ_CheckSeekTableCRCs
      •  OodleLZ_FindSeekEntry
      •  OodleLZ_GetSeekEntryPackedPos
      •  OodleLZ_CompressionLevel_GetName
      •  OodleLZ_Compressor_GetName
      •  OodleLZ_Jobify_GetName
      •  OodleLZ_CompressOptions_GetDefault
      •  OodleLZ_CompressOptions_Validate
      •  OodleLZ_Compressor_UsesWholeBlockQuantum
      •  OodleLZ_Compressor_UsesLargeWindow
      •  OodleLZ_Compressor_CanDecodeInCircularWindow
      •  OodleLZ_Compressor_CanDecodeThreadPhased
      •  OodleLZ_Compressor_CanDecodeInPlace
      •  OodleLZ_Compressor_MustDecodeWithoutResets
      •  OodleLZ_Compressor_CanDecodeFuzzSafe
      •  OodleLZ_Compressor_RespectsDictionarySize
      •  OodleLZ_GetCompressScratchMemBound
      •  OodleLZ_GetCompressScratchMemBoundEx
      •  OodleLZ_GetCompressedBufferSizeNeeded
      •  OodleLZ_GetDecodeBufferSize
      •  OodleLZ_GetInPlaceDecodeBufferSize
      •  OodleLZ_GetCompressedStepForRawStep
      •  OodleLZ_GetAllChunksCompressor
      •  OodleLZ_GetFirstChunkCompressor
      •  OodleLZ_GetChunkCompressor
    •  Typedefs
      •  OodleDecompressCallback

 
OodleCore_Plugins_SetAssertionOodle2 Core API DocumentationAbout OodleLZ

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleDecompressCallback_WriteFile_Data
Navigation
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleDecompressCallback_WriteFile_Data
{
    OodleXIOQFile   file;
    OodleXHandle    lastWriteH;
    OodleXHandle    closeH;
    OO_SINTa        written;
    OO_BOOL         doCloseFile;
};
Discussion
A OodleDecompressCallback_WriteFile_Data for use with OodleDecompressCallback_WriteFile
Members
file  the file handle to write to
lastWriteH  handle to the last write operation ; it's autodelete
closeH  handle to the file close operation ; NOT autoDelete
written  number of bytes written so far
doCloseFile  should the file be closed after the last write?
Discussion
The OodleDecompressCallback_WriteFile_Data struct is passed as "userdata" to OodleDecompressCallback_WriteFile.

You must supply one as pcbData in functions that take a decompression callback.

Warning : if you make this object on the stack, ensure the lifetime is sufficient for the async operation!
 

OodleXAPI_LZ_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Decompress_ThreadPhased_Narrow_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressOptions_GetDefault
Navigation
 OodleLZ_Jobify_GetName
 OodleLZ_CompressOptions_Validate
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleLZ_CompressOptions * OodleLZ_CompressOptions_GetDefault( OodleLZ_Compressor compressor OODEFAULT( OodleLZ_Compressor_Invalid ),
                                                                    OodleLZ_CompressionLevel lzLevel OODEFAULT( OodleLZ_CompressionLevel_Normal ) );
Discussion
Provides a pointer to default compression options
Parameters
compressor  deprecated, ignored
lzLevel  deprecated, ignored
Discussion

Use to fill your own OodleLZ_CompressOptions then change individual fields.


 

OodleLZ_Jobify_GetNameOodleAPI_LZ_CompressorsOodleLZ_CompressOptions_Validate

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleLZ ThreadPhased Decode
Navigation
 About OodleLZ
 Core LZ compression
 About OodleLZ Hydra
 Welcome to Oodle
 Change Log

About OodleLZ ThreadPhased Decode

The new Kraken compressor can be decoded with a new type of parallelism called "ThreadPhased". (check OodleLZ_Compressor_CanDecodeThreadPhased)

ThreadPhased decoding works by running the decode operation per block in two phases. This allows 1-2X speedup using two threads for decoding, typically around 33%-50% speedup. (Mermaid can decode ThreadPhased too, but the benefit isn't as large as it is with Kraken)

ThreadPhased decoding can be done on the normal compressed data made with OodleLZ_Compress. You don't need to prepare the data specially for ThreadPhased decoding, or break it into chunks with the Oodle seek chunk reset system. (you can combine seek chunks and ThreadPhased decoding for even more parallelism if you like).

Make compressed data by calling OodleLZ_Compress just like you normally would.

ThreadPhased decoding requires more memory than normal single-threaded decoding, because it needs staging space for the two threads to communicate.

The easiest way to try ThreadPhased decoding is to use the helper in OodleX OodleXLZ_Decompress_ThreadPhased_Narrow_Async . That runs a 2-thread decode on the OodleX Worker system, freeing the calling thread for other work.

example_lz_threadphased : Example of 2-thread ThreadPhased decoding includes a demonstration of that call.

The basic idea of ThreadPhased decoding is that the OodleLZ_Decompress work on each BLOCK can be split into two phases. This can be invoked by just calling OodleLZ_Decompress twice on the same block, first with OodleLZ_Decode_ThreadPhase1, then with OodleLZ_Decode_ThreadPhase2.

To get parallelism, we can run the two phases on two separate threads.

The rule is that you must run the Phase2 on each block after the Phase1 for that block is done, and with the same "decoderMem" pointer. The Phase2 decodes on all blocks must be done in sequential order (unless they are Seek Resets). The Phase1 decodes can be done in any order. The decoder memory used for OodleLZ_Decompress here must be larger than normal, of size OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded().

To perform a portion of a ThreadPhased decode, you simply call OodleLZ_Decompress , but with some special argument values :

OodleLZ_Decompress( compBuf,compBufferSize, // should be the compressed data for one block only rawBuf,rawLen, // should be the destination uncompressed pointers for this block checkCRc,verbosity, decBufBase,decBufSize, // should be the whole destination buffer (or one whole seek chunk) fpCallback,callbackUserData, decoderMemory,decoderMemorySize, // must be provided, of size OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded() threadPhase // set to OodleLZ_Decode_ThreadPhase1 or OodleLZ_Decode_ThreadPhase2 );

The decoderMemory must be allocated by the caller, and must be the same for each Phase on the block. That is, don't use the decoderMemory for the next block in the same phase, instead use it for the same block in the next phase. The decoderMemory is the place where the work of phase1 is passed to phase2.

The two threads run through the blocks sequentially, passing the results of phase1 to the input of phase2. Each phased block must get its own decoderMemory.


[phase1 block1] [phase1 block2] ...							
              |               |
              |               |
              [phase2 block1] [phase2 block2] ...

See example_lz_threadphased : Example of 2-thread ThreadPhased decoding for a full client-side implementation
 

About OodleLZCore LZ compressionAbout OodleLZ Hydra

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Windows UWP
Navigation
 About Oodle on Windows
 Welcome to Oodle
 Change Log

Oodle for Windows UWP contains the Core module only (no ext) and is provided as a DLL.

Oodle for Windows UWP is built with MSVC 2017 and the Windows 10 SDK.

lib/oo2core_winuwp32.lib
lib/oo2core_winuwp64.lib
redist/oo2core_5_winuwp32.dll
redist/oo2core_5_winuwp64.dll

The debug version of the DLL is also provided to help during development. It should not be redistributed.

C++ apps for UWP can use the import libs to access the DLL.

C# and Store apps may need to use the "LoadPackagedLibrary" mechanism. What you do is add the DLL to the project as content (and make sure to check "deploy" in its properties). Then you load it with "LoadPackagedLibrary" (not the normal LoadLibrary) to get an HMODULE , then you can do GetProcAddress to get func pointers.

If your UWP client has very limited Oodle usage, you could get away with only importing the OodleLZ_Decompress function.

(extra step: if the UWP app is C#, not C++, then you must also tell it to deploy the VC runtime which it will not do by default)
 

About Oodle on WindowsAbout Oodle on WindowsAbout Oodle on PS4

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_SetFailedHandler
Navigation
 OodleXMalloc_InstallVTable
 OodleXMalloc
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXMalloc_SetFailedHandler( OodleXMallocFailedHandler * f );
Discussion
Install the OodleXMallocFailedHandler that will be used
Parameters
f  the function pointer to call (can be null for none)
Discussion


 

OodleXMalloc_InstallVTableOodleX Memory AllocatorsOodleXMalloc

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_UsesLargeWindow
Navigation
 OodleLZ_Compressor_UsesWholeBlockQuantum
 OodleLZ_Compressor_CanDecodeInCircularWindow
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_UsesLargeWindow( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor is "LargeWindow" or not, meaning it can benefit from a Long-Range-Matcher and windows larger than OODLELZ_BLOCK_LEN
 
OodleLZ_Compressor_UsesWholeBlockQuantumOodleAPI_LZ_CompressorsOodleLZ_Compressor_CanDecodeInCircularWindow

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Jobify
Navigation
 OodleLZ_CompressionLevel
 OodleLZ_Decode_ThreadPhase
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_Jobify
{
    OodleLZ_Jobify_Default = 0,
    OodleLZ_Jobify_Disable = 1,
    OodleLZ_Jobify_Normal = 2,
    OodleLZ_Jobify_Aggressive = 3,
    OodleLZ_Jobify_Count = 4,
    OodleLZ_Jobify_Force32 = 0x40000000
};
Discussion
Controls the amount of internal threading in OodleLZ_Compress calls
Enumerants
OodleLZ_Jobify_Default  Use compressor default for level of internal job usage
OodleLZ_Jobify_Disable  Don't use jobs at all
OodleLZ_Jobify_Normal  Try to balance parallelism with increased memory usage
OodleLZ_Jobify_Aggressive  Maximize parallelism even when doing so requires large amounts of memory
OodleLZ_Jobify_Count 
OodleLZ_Jobify_Force32 
Discussion
Once you install a pluggable job system via OodleCore_Plugins_SetJobSystem, Oodle can internally break heavy-weight compression tasks into smaller jobs that can run in parallel. This can speed up compression of large blocks of data at Optimal1 and higher levels substantially.

The trade-off is that running more jobs concurrently rather than sequentially can greatly increase memory requirements when there are multiple outstanding memory-intensive jobs.

OodleLZ_Jobify_Default lets the compressor decide; typically compressors will default to "Normal" when a pluggable job system has been installed, and "Disable" otherwise.

OodleLZ_Jobify_Disable disables use of internal jobs entirely; all compression work is done on the calling thread. This minimizes the amount of memory used, and is also appropriate when you're getting parallelism in other ways, e.g. by running OodleLZ_Compress on many threads yourself.

OodleLZ_Jobify_Normal uses jobs to increase compressor parallelism and speeds up compression of large blocks of data, but avoids handing out many concurrent jobs for tasks that are memory-intensive.

OodleLZ_Jobify_Aggressive will use concurrent jobs even for highly memory-intensive tasks. This can speed up things further, but at a potentially significant increase in the amount of memory used by Oodle.


 

OodleLZ_CompressionLevelOodleAPI_LZ_CompressorsOodleLZ_Decode_ThreadPhase

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_SetFileSize_Async
Navigation
 OodleXIOQ_Write_Async
 OodleXIOQ_ReserveFileSizeForWrite_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_SetFileSize_Async( OodleXIOQFile file,
                                          OO_S64 size,
                                          OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                          OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                          const OodleXHandle * dependencies OODEFAULT( NULL ),
                                          OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a set-file-size request
Parameters
file  the file to act on
size  the new file size
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Change the size of a file open for writing.

OodleXIOQ_SetFileSize_Async may align up size to the next sector boundary or OODLEX_IO_MAX_ALIGNMENT. The contents of the file in the resized but unwritten area are undefined/garbage.

To write a file with non-aligned size, use OodleXIOQ_CloseFile_Async:truncateFileSize in OodleXIOQ_CloseFile_Async.

If the purpose of calling SetFileSize is to pre-reserve space to make writes go faster, then use OodleXIOQ_ReserveFileSizeForWrite_Async instead.
 

OodleXIOQ_Write_AsyncOodleX low level async ioOodleXIOQ_ReserveFileSizeForWrite_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_WaitDoneAllPending
Navigation
 OodleX_WaitAll
 OodleX_SetHandleAutoDelete
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_WaitDoneAllPending( );
Discussion
Block on all pending operations being completed
Discussion
FlushAllAsync kills all parallelism and should generally only be used at shutdown or error handling.

FlushAllAsync is only guaranteed to stop pending handles that were fired before this call starts. If new operations are created by other threads (or by existing pending operations) they may still be pending when this call returns.
 

OodleX_WaitAllOodleX async handle operationsOodleX_SetHandleAutoDelete

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: I write a file with IOQ but the contents are garbage?
Navigation
 FAQ: How do I get the Oodle logs?
 Frequently Asked Questions
 FAQ: I ran out of OodleHandle table slots; what do I do?
 Welcome to Oodle
 Change Log

A very common error is using an async write, but failing to keep the buffer alive long enough.

For example, code like this :


	OodleIOQFile file; // parameter
	SINTa size; // parameter

	void * buffer = OodleXMalloc_IOAligned(size);
	// .. fill buffer ..

	SINTa alignedSize = OodleAlignUp(size);
	
	OodleIOQ_Write_Async(toFile,buffer,alignedSize,0,OodleXHandleAutoDelete_Yes);
	
	OodleFree_IOAligned(buffer);

may free the buffer during the async IO. This may or may not produce an error. In some cases the write will succeed but simply write junk data. (in some cases the write may succeed and write the buffer correctly; this is a race condition and the results are unpredictable).

To fix this you must ensure the buffer is not freed until the write is done. Often you will want to OodleX_Wait on the write being done for some other reason, so you can simply move the buffer free until after the Wait.

Another option is to enqueue the Free using OodleXIOQ_FreeBufferIOAligned_Async.
 

FAQ: How do I get the Oodle logs?Frequently Asked QuestionsFAQ: I ran out of OodleHandle table slots; what do I do?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXUtil_ConvertUTF16ToUTF8
Navigation
 OodleXUtil_ConvertUTF8ToUTF16
 OodleX_CombinePaths
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXUtil_ConvertUTF16ToUTF8( const OO_U16 * from,
                                       char * to,
                                       int toSize );
Discussion
Convert a UTF16 string to UTF8
Parameters
from  string to convert
to  filled with the result
toSize  number of characters availabe in to (not bytes!)
Discussion

Fill [ to , to + toSize ] with the UTF8 conversion of from (UTF16).

Oodle strings are all UTF8. This can be used to convert a string from UTF16 , for example for use with the Windows wchar routines.

Note that OodleXUtil_ConvertUTF8ToUTF16 + OodleXUtil_ConvertUTF16ToUTF8 may not produce the same output, because UTF encodings are not unique. Also note that Windows 8-bit names (from "A" code page, though they are not ANSI or ASCII) are not UTF8.
 

OodleXUtil_ConvertUTF8ToUTF16OodleX UtilsOodleX_CombinePaths

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Jobify_GetName
Navigation
 OodleLZ_Compressor_GetName
 OodleLZ_CompressOptions_GetDefault
 Welcome to Oodle
 Change Log
// Function prototype:
const char * OodleLZ_Jobify_GetName( OodleLZ_Jobify jobify );
Discussion
Provides a string naming a OodleLZ_Jobify enum
 
OodleLZ_Compressor_GetNameOodleAPI_LZ_CompressorsOodleLZ_CompressOptions_GetDefault

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_SetEcho
Navigation
 OodleXLog_SetState
 OodleXLog_GetEcho
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXLog_SetEcho( void * echo );
Discussion
Set the echo output file
Parameters
echo  the stdio FILE to output to
Discussion

You must also enable the flag OODLEXLOG_ECHO in OodleXLog_SetState if you want output to the echo FILE.

You can call SetEcho(NULL) to disable echoing, or you can SetState and disable the OODLEXLOG_ECHO flag.
 

OodleXLog_SetStateOodleX Debug aidsOodleXLog_GetEcho

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetOSCwd
Navigation
 OodleX_CombinePaths
 OodleX_PrefixOSCwd
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_GetOSCwd( char * into,
                      OO_S32 intoSize );
Discussion
Get current working directory
Parameters
into  filled with cwd (OS,UTF8)
intoSize  size of into
Discussion

After the call, into ends in a path delim, so file names can be stuck on it with strcat.
 

OodleX_CombinePathsOodleX UtilsOodleX_PrefixOSCwd

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXOSFile
Navigation
 OodleXIOQFile
 OodleXOSFileListing
 Welcome to Oodle
 Change Log
// Type definition:
void * OodleXOSFile
Discussion
OS file handle abstract type
 
OodleXIOQFileOodleX low level async ioOodleXOSFileListing

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_ThreadProfilerInit
Navigation
 OodleX_Init_GetDefaults
 Welcome to Oodle
 Change Log
// Function prototype:
t_OodleFPVoidVoid * OodleX_Init_ThreadProfilerInit( void );
Discussion
Get the func pointer for m_OodleInit_ThreadProfiler_funcptr
 
OodleXConfigValuesOodleX Startup and ShutdownOodleX_Init_GetDefaults

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
Navigation
 OODLEX_FILE_OPEN_NO_RESERVE_SIZE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE (-1)
Discussion
Pass for truncateFileSize to OodleXIOQ_CloseFile_Async if you don't want it to truncate
 
OODLEX_FILE_OPEN_NO_RESERVE_SIZEOodleX low level async ioOodleXCopyFileFlags

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_BUFFER_SIZE_DEFAULT
Navigation
 OODLEX_IO_MAX_ALIGNMENT
 OODLEX_FILEINFO_FLAG_INVALID
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_BUFFER_SIZE_DEFAULT (-1)
Discussion
Pass to functions that want a buffer size to indicate the default should be used.
Discussion
The buffer size used comes from OodleXConfigValues
 
OODLEX_IO_MAX_ALIGNMENTOodleX low level async ioOODLEX_FILEINFO_FLAG_INVALID

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXDecompressCallback_WriteFile_Data_Init
Navigation
 OodleXLZ_ReadAndDecompress_Stream_Async
 OodleDecompressCallback_WriteFile
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXDecompressCallback_WriteFile_Data_Init( OodleDecompressCallback_WriteFile_Data * pcbData,
                                                   const char * fileName,
                                                   OO_BOOL closeFileAfterWriting,
                                                   OO_SINTa reserveSize OODEFAULT( 0 ) );
Discussion
fills out an OodleDecompressCallback_WriteFile_Data struct
Parameters
pcbData  The OodleDecompressCallback_WriteFile_Data to fill
fileName  The name of the file to write to (will be opened)
closeFileAfterWriting  Should the file be closed for you after the last write
reserveSize  (optional) size to reserve
Discussion

Fills out the pcbData for use with OodleDecompressCallback_WriteFile.

Opens fileName for write with OodleXIOQ_OpenForWriteCreate_Async.


 

OodleXLZ_ReadAndDecompress_Stream_AsyncOodleXAPI_LZ_AsyncOodleDecompressCallback_WriteFile

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_ReleaseThreadTLS
Navigation
 OodleX_WaitAndDestroyThread
 OodleX_CorePlugin_RunJob
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_ReleaseThreadTLS( );
Discussion
Release OodleX TLS resources on the calling thread
Discussion
Call on a thread before it terminates to release resources that OodleX may have put in the TLS of this thread.

The purpose of this is to avoid increasing memory use in code bases that create & destroy a lot of threads for jobs. In that case, Oodle may allocate a bit of memory per thread and never free it, which will add up over time.

In normal game code bases that create a fixed number or low number of threads, you should not bother calling this.

NOTE : any use of OodleX functions on this thread after calling this may crash! This should be the last thing called on this thread before it terminates or returns from its thread function.
 

OodleX_WaitAndDestroyThreadOodleX threading utilOodleX_CorePlugin_RunJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_KickAnyDelayed
Navigation
 OodleXIOQ_WaitDoneAllPending
 OodleXIOQ_GetStatus
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXIOQ_KickAnyDelayed( );
Discussion
Fire any requests which have not previously been started
Discussion
If requests were enqueued with kick = false (don't start immediately), then they can be started this way. Disabling auto-kick is good for performance when a very large number of request are being created in a short period of time.
 
OodleXIOQ_WaitDoneAllPendingOodleX low level async ioOodleXIOQ_GetStatus

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetExtensionKey
Navigation
 OodleX_MakeExtensionKey
 Welcome to Oodle
 Change Log
// Function prototype:
OO_U32 OodleX_GetExtensionKey( const char * filename );
Discussion
convert the file name extension (max 4 chars) into a U32 for fast compares
Parameters
filename  file name with an extension after a period
Return Value
return  a U32 with up to 4 chars of extension
Discussion

The return value is case-insensitive and does not include the dot.
 

OODLEX_PATH_DELIMOodleX UtilsOodleX_MakeExtensionKey

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_StateCompacted_MaxSize
Navigation
 OodleNetwork1UDP_Decode
 OodleNetwork1UDP_State_Compact_ForVersion
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1UDP_StateCompacted_MaxSize( void );
Discussion
Returns the size of memory required for an OodleNetwork1UDP_StateCompacted object
Discussion
Shared and State are allocated with malloc( Size() )


 

OodleNetwork1UDP_DecodeOodleAPI_OodleNetwork1OodleNetwork1UDP_State_Compact_ForVersion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CheckSeekTableCRCs
Navigation
 OodleLZ_FreeSeekTable
 OodleLZ_FindSeekEntry
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_CheckSeekTableCRCs( const void * rawBuf,
                                    OO_SINTa rawLen,
                                    const OodleLZ_SeekTable * seekTable );
Discussion
Check the CRC's in seekTable vs rawBuf
Parameters
rawBuf  uncompressed buffer
rawLen  size of rawBuf
seekTable  result of OodleLZ_CreateSeekTable
Return Value
return  true if the CRC's check out
Discussion

Note that OodleLZ_Decompress option of OodleLZ_CheckCRC checks the CRC of compressed data, this call checks the CRC of the raw (uncompressed) data.

OodleLZ data contains a CRC of the compressed data if it was made with OodleLZ_CompressOptions:sendQuantumCRCs. The SeekTable contains a CRC of the raw data if it was made with OodleLZSeekTable_Flags_MakeRawCRCs.

Checking the CRC of compressed data is faster, but does not verify that the decompress succeeded.
 

OodleLZ_FreeSeekTableOodleAPI_LZ_CompressorsOodleLZ_FindSeekEntry

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State_Compact_ForVersion
Navigation
 OodleNetwork1UDP_StateCompacted_MaxSize
 OodleNetwork1UDP_State_Compact
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1UDP_State_Compact_ForVersion( OodleNetwork1UDP_StateCompacted * to,
                                                    const OodleNetwork1UDP_State * from,
                                                    OO_S32 for_oodle_major_version );
Discussion
See OodleNetwork1UDP_State_Compact * * takes oodle_major_version to target * * Oodle Network Compacted state changed from major version 5 to 6 (eg 2.5.5 to 2.6.0) *
 
OodleNetwork1UDP_StateCompacted_MaxSizeOodleAPI_OodleNetwork1OodleNetwork1UDP_State_Compact

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetCompressScratchMemBound
Navigation
 OodleLZ_Compressor_RespectsDictionarySize
 OodleLZ_GetCompressScratchMemBoundEx
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetCompressScratchMemBound( OodleLZ_Compressor compressor,
                                             OodleLZ_CompressionLevel level,
                                             OO_SINTa rawLen,
                                             const OodleLZ_CompressOptions * pOptions OODEFAULT( NULL ) );
Discussion
Return the maximum amount of scratch mem that will be needed by OodleLZ_Compress
Parameters
compressor  which OodleLZ variant to use in compression
level  OodleLZ_CompressionLevel controls how much CPU effort is put into maximizing compression
rawLen  maximum number of bytes you will compress (plus dictionary backup)
pOptions  (optional) options; if NULL, OodleLZ_CompressOptions_GetDefault is used
Discussion

If you pass scratch mem to OodleLZ_Compress of this size, it is gauranteed to do no allocations. (normally if it runs out of scratch mem, it falls back to the installed allocator)

For rawLen pass at least the maximum size you will ever encode. If your data is divided into chunks, pass the chunk size. If you will encode full buffers of unbounded size, pass -1.

The options must be the same as when you call OodleLZ_Compress

Some options and levels may not have simple finite bounds. Then OODLELZ_SCRATCH_MEM_NO_BOUND is returned and the call to OodleLZ_Compress may use the allocator even if infinite scratch memory is provided. Currently this applies to all the Optimal levels.

When OODLELZ_SCRATCH_MEM_NO_BOUND is returned, you can still pass in scratch mem which will be used before going to the plugin allocator.


 

OodleLZ_Compressor_RespectsDictionarySizeOodleAPI_LZ_CompressorsOodleLZ_GetCompressScratchMemBoundEx

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_MakeAllDirs_AsyncAndWait
Navigation
 OodleXIOQ_SetInfoByName_AsyncAndWait
 OodleXIOQ_Delete_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_MakeAllDirs_AsyncAndWait( const char * path );
Discussion
See OodleXIOQ_MakeAllDirs_Async
 
OodleXIOQ_SetInfoByName_AsyncAndWaitOodleX low level async ioOodleXIOQ_Delete_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNet_Plugins_SetJobSystem
Navigation
 OodleNet_Plugins_SetAllocators
 OodleNet_Plugins_SetJobSystemAndCount
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNet_Plugins_SetJobSystem( t_fp_OodleNet_Plugin_RunJob * fp_RunJob,
                                    t_fp_OodleNet_Plugin_WaitJob * fp_WaitJob );
Discussion
DEPRECATED use OodleNet_Plugins_SetJobSystemAndCount instead
Discussion
See OodleNet_Plugins_SetJobSystemAndCount
 
OodleNet_Plugins_SetAllocatorsNetwork pluginsOodleNet_Plugins_SetJobSystemAndCount

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Capturing Training data for OodleNetwork
Navigation
 About Oodle Network Compression
 Forming Packets for Maximum Compression
 Welcome to Oodle
 Change Log

To evaluate Oodle Network fairly, you will need to capture network packets from a real play session.

Once you are using Oodle Network in production, you will need to make a large capture to train the compressor that you ship.

If possible, capture from a real game session (perhaps from your QA department playing the game), not a simulation using bots. Simulated captures can have patterns that don't reflect real play.

The captured packets should be without any encryption or other compression algorithms applied. Any already compressed data (such as zlib or jpeg or voice data) should be excluded from the packet capture. Very large packets that you will compress with OodleLZ should also be excluded.

You should typically continue to use your heuristic bit-packing or delta scheme.

Try to capture at least 100 MB of packet data for the evalution. For final game training you should capture more. Note that just capturing a very long single session is not generally helpful; you want to take a broad sampling of many sessions over time to ensure that the capture is reflective of the whole spectrum of packets that the game sends.

The better the training packets match the packets seen in the final game, the more compression there will be. Mismatches are not a disaster, they simply mean that part of the dictionary is not useful for compression.

The example_packet : Example demonstrating network packet compression shipped with Oodle reads this file format :

packet.bin :
U32 [LE] : numbers of channels (num_channels)
repeatedly :
{
	U32 [LE] : channel index in [0,num_channels-1]
	U32 [LE] : number of bytes of data in this packet (num_bytes)
	U8 * num_bytes : payload of this packet
}

though you may always change example_packet to read a different format.

For UDP, write num_channels = 1 and the channel index of all packets as 0.

The cleanest way to capture packets is to add code to your server to log them out immediately before sending. If you can't change your server code, then something like tcpdump can be used, but you will have to strip the protocol headers.

For your final shipping capture, you should try to capture packets from a wide variety of play sessions in different levels, with different numbers of players, to get a broad sampling of what your network traffic looks like. Then combine random portions of those captures to make the packet file that you use for Oodle Network training. This ensures that you don't train on a non-representative set of data.

NOTE : the packets that you hold out for testing/training/dictionary should be a random selection of packets, not linear chunks. You want each group of packets to be an independent random sampling of the network traffic. Each group should span the range of different types of data you send. (example_packet : Example demonstrating network packet compression includes one way of doing this)

NOTE : if you are compressing both upstream and downstream, those should generally have different dictionaries and different trained states. Typically the nature of the network traffic up and down is very different, so they should not be mixed together in a single capture. If you have very distinct network traffic phases (such as, for example, a lobby or match-making phase and then a match play phase) then it may be advantageous to separate those types of traffic for compression.

Contact oodle@radgametools.com with any questions
 

About Oodle Network CompressionAbout Oodle Network CompressionForming Packets for Maximum Compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXStatus
Navigation
 OodleXAsyncSelect
 OodleXHandleAutoDelete
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXStatus
{
    OodleXStatus_Invalid = 0,
    OodleXStatus_Pending = 1,
    OodleXStatus_Done = 2,
    OodleXStatus_Error = 3,
    OodleXStatus_Count = 4,
    OodleXStatus_Force32 = 0x40000000
};
Discussion
OodleXStatus indicates the status of asynchronous weak reference handles.
Enumerants
OodleXStatus_Invalid  indicates that a handle is not a live object (possibly previously deleted)
OodleXStatus_Pending  handle is alive and pending
OodleXStatus_Done  handle completed succesfully
OodleXStatus_Error  handle completed in error state
OodleXStatus_Count 
OodleXStatus_Force32 
Discussion
The OodleXStatus generally increases in numeric value during its autoDelete. Check status >= OodleXStatus_Done to test for completion (possibly error).

	Not yet allocated : OodleXStatus_Invalid = 0
	Fired off and still pending : OodleXStatus_Pending = 1
	Completed (possibly in error) : OodleXStatus_Done = 2 or Error = 3
	

 
OodleXAsyncSelectOodleX async handle operationsOodleXHandleAutoDelete

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_StateCompacted
Navigation
 OodleNetwork1UDP_State
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleNetwork1UDP_StateCompacted;
Discussion
Opaque type for an OodleNetwork1UDP_StateCompacted
Discussion
Compacted version of OodleNetwork1UDP_State

Used to decrease the size of OodleNetwork1UDP_State for storage. You cannot code with a OodleNetwork1UDP_StateCompacted.
 

OodleNetwork1UDP_StateOodleAPI_OodleNetwork1OodleNetwork1_Shared_Size

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX LZ compression
Navigation
 Oodle2 Ext API Documentation
 OodleX Startup and Shutdown
 Welcome to Oodle
 Change Log

Oodle2 Ext LZ lossless data compression.

  •  OodleXAPI_LZ_Async
    •  Structures
      •  OodleDecompressCallback_WriteFile_Data
    •  Functions
      •  OodleXLZ_Decompress_ThreadPhased_Narrow_Async
      •  OodleXLZ_Decompress_Narrow_Async
      •  OodleXLZ_ReadAndDecompress_Wide_Async
      •  OodleXLZ_Decompress_Wide_Async
      •  OodleXLZ_Decompress_MakeSeekTable_Wide_Async
      •  OodleXLZ_Compress_Async
      •  OodleXLZ_Compress_Wait_GetResult
      •  OodleXLZ_Compress_AsyncAndWait
      •  OodleXLZ_ReadAndDecompress_Stream_Async
      •  OodleXDecompressCallback_WriteFile_Data_Init
      •  OodleDecompressCallback_WriteFile

 
Oodle2 Ext API DocumentationOodle2 Ext API DocumentationOodleXAPI_LZ_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Acknowledgements
Navigation
 Welcome to Oodle
 Change Log

Acknowledgements

The product names Oodle, Bink, Miles, Iggy, Granny and Telemetry are all copyrighted and trademarked by Epic Games Tools LLC. Printed and produced in the United States of America.

As a licensee of Oodle, you must abide by the terms set forth in your license agreement. Please refer to that agreement if you have any questions about what you may or may not do with this documentation or the software to which it pertains. Oodle is not copy protected, but it is copyrighted. We think our license agreements are fair, and that our software is reasonably priced for the quality and effort we have put into it. Using our software in ways other than allowed by your license agreement violates federal, civil, and criminal law. We rely primarily on your good faith not to violate our copyright; please respect it.

This software and documentation are provided "as is" without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. In no event will Epic Games Tools LLC be liable to you for damages, including any general, special, incidental or consequential damages arising out of the use or inability to use the product (including, but not limited to, loss of data).

Oodle was written by Charles Bloom and Fabian Giesen.

The IDOC automatic documentation system used for Oodle was written by Sean Barrett.

The compression algorithms in Oodle benefitted from discussion with countless developers who have shared their thoughts and code with me over the years.


  •  Third Party License Notices

 
  Third Party License Notices

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
Navigation
 OODLENETWORK1_HASH_BITS_DEFAULT
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN (5)
Discussion
compressed buffer must be sized to at least compLen+OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
Discussion
(note that this is strictly less than OodleNetwork1_CompressedBufferSizeNeeded(rawLen))


 

OODLENETWORK1_HASH_BITS_DEFAULTOodleAPI_OodleNetwork1OodleNetwork1_Shared

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_MallocAligned
Navigation
 t_fp_OodleCore_Plugin_Free
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void *( OODLE_CALLBACK t_fp_OodleCore_Plugin_MallocAligned )( OO_SINTa bytes,
                  OO_S32 alignment );
Discussion
Function pointer type for OodleMallocAligned
Parameters
bytes  number of bytes to allocate
alignment  required alignment of returned pointer
Return Value
return  pointer to memory allocated (must not be NULL)
Discussion

alignment will always be a power of two

alignment will always be >= OODLE_MALLOC_MINIMUM_ALIGNMENT


 

OodleCore_Plugins_SetAssertionCore pluginst_fp_OodleCore_Plugin_Free

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_Decode
Navigation
 OodleNetwork1TCP_Encode
 OodleNetwork1UDP_Train
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1TCP_Decode( OodleNetwork1TCP_State * state,
                                 const OodleNetwork1_Shared * shared,
                                 const void * comp,
                                 OO_SINTa compLen,
                                 void * raw,
                                 OO_SINTa rawLen );
Discussion
Decode a packet
Parameters
state  state of this compression channel; will be mutated
shared  const shared compression context
comp  compressed packet received
compLen  size of compressed data
raw  output decompressed packet
rawLen  size of the packet to write
Return Value
return  false for failure
Discussion

Decodes one packet. state is mutated, learning from this packet for future packets.

The rawLen provided here must match the length used in OodleNetwork1TCP_Encode when creating this compressed packet. The OodleNetwork1 data is headerless, it's up to you to send the packet decompressed size in your own header.

If corrupt data is detected, false is returned.

If the number of compressed bytes consumed does not match compLen, false is returned. If the number of output bytes does not match rawLen, false is returned.

This function, however, does not do verify data integrity. It will return 'true' if the correct number of bytes are coded, even if the data does not match.

The buffer comp must be allowed to read at least compLen + OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN bytes (note that this is strictly less than OodleNetwork1_CompressedBufferSizeNeeded(rawLen) , so that may be used as well)


 

OodleNetwork1TCP_EncodeOodleAPI_OodleNetwork1OodleNetwork1UDP_Train

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXOSFileListing
Navigation
 OodleXOSFile
 Welcome to Oodle
 Change Log
// Type definition:
void * OodleXOSFileListing
Discussion
OS file listing abstract type
 
OodleXOSFileOodleX low level async ioOodleX Debug aids

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_LOCALDICTIONARYSIZE_MAX
Navigation
 OODLE_ALLOW_DEPRECATED_COMPRESSORS
 OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_LOCALDICTIONARYSIZE_MAX (1<<30)
Discussion
Maximum value of maxLocalDictionarySize in OodleLZ_CompressOptions
 
OODLE_ALLOW_DEPRECATED_COMPRESSORSOodleAPI_LZ_CompressorsOODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenAndReadMallocWholeFile_Async
Navigation
 OodleXIOQ_ReadMallocWholeFile_GetResult
 OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenAndReadMallocWholeFile_Async( OodleXIOQFile * pFile,
                                                         const char * name,
                                                         OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                         const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                         OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                         OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                         const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                         OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to open a file, allocate a buffer for a whole file and read it
Parameters
pFile  filled with a handle to the file which will be opened
name  name of the file to open (VFS, UTF-8)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
Return Value
return  handle to the RMWF op; use OodleXIOQ_ReadMallocWholeFile_GetResult
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

Performs OodleXIOQ_OpenForRead_Async and OodleXIOQ_ReadMallocWholeFile_Async.

The OodleXHandle returned is to the RMWF operation; use OodleXIOQ_ReadMallocWholeFile_GetResult.

You will normally want to enqueue an OodleXIOQ_CloseFile_Async after this.
 

OodleXIOQ_ReadMallocWholeFile_GetResultOodleX low level async ioOodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_Destroy
Navigation
 OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
 OodleLZDecoder_Reset
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleLZDecoder_Destroy( OodleLZDecoder * decoder );
Discussion
Pairs with OodleLZDecoder_Create
Discussion
You should always call Destroy even if you provided the memory for OodleLZDecoder_Create
 
OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeededOodleAPI_LZ_CompressorsOodleLZDecoder_Reset

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILEINFO_FLAGS
Navigation
 OodleFileNotFoundIsAnError
 OodleXError
 Welcome to Oodle
 Change Log
// Enumerant:
enum OODLEX_FILEINFO_FLAGS
{
    OODLEX_FILEINFO_FLAG_DIR = (1<<0),
    OODLEX_FILEINFO_FLAG_READONLY = (1<<1),
    OODLEX_FILEINFO_FLAG_HIDDEN = (1<<2),
    OODLEX_FILEINFO_FLAG_SYMLINK = (1<<3),
    OODLEX_FILEINFO_FLAG_TEMPORARY = (1<<4),
    OODLEX_FILEINFO_FLAG_OFFLINE = (1<<5),
    OODLEX_FILEINFO_FLAG_Force32 = 0x40000000
};
Discussion
Flags for OodleXFileInfo:flags
Enumerants
OODLEX_FILEINFO_FLAG_DIR  queried name is a directory
OODLEX_FILEINFO_FLAG_READONLY  you do not have write permission for this file
OODLEX_FILEINFO_FLAG_HIDDEN  file is marked hidden
OODLEX_FILEINFO_FLAG_SYMLINK  file or dir is a symlink or reparse point
OODLEX_FILEINFO_FLAG_TEMPORARY  file is marked temporary
OODLEX_FILEINFO_FLAG_OFFLINE 
OODLEX_FILEINFO_FLAG_Force32 

 
OodleFileNotFoundIsAnErrorOodleX low level async ioOodleXError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_Encode
Navigation
 OodleNetwork1TCP_Train
 OodleNetwork1TCP_Decode
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1TCP_Encode( OodleNetwork1TCP_State * state,
                                  const OodleNetwork1_Shared * shared,
                                  const void * raw,
                                  OO_SINTa rawLen,
                                  void * comp );
Discussion
Encode a packet
Parameters
state  state of this compression channel; will be mutated
shared  const shared compression context
raw  packet bytes to compress
rawLen  size of the packet to compress ; can be >= 0
comp  output compressed bytes; must be allocated to at least OodleNetwork1_CompressedBufferSizeNeeded bytes
Return Value
return  length of output compressed data written to comp ; the returned compLen is strictly <= rawLen
Discussion

Encodes one packet. state is mutated, learning from this packet for future packets.

The returned compLen will never be greater than rawLen, because OodleNetwork1 won't send packets that expand under compression (it just sends them uncompressed) - however it may write further than that during the compression attempt. Do not use the returned compLen to check the size of the compressed buffer needed.


 

OodleNetwork1TCP_TrainOodleAPI_OodleNetwork1OodleNetwork1TCP_Decode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ReserveFileSizeForWrite_Async
Navigation
 OodleXIOQ_SetFileSize_Async
 OodleXIOQ_ForceWriteable_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_ReserveFileSizeForWrite_Async( OodleXIOQFile file,
                                                      OO_S64 size,
                                                      OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                      OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                      const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                      OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a set-file-size request, if it helps write speed.
Parameters
file  the file to act on
size  the new file size
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

If the purpose of calling SetFileSize is to pre-reserve space to make writes go faster, then use OodleXIOQ_ReserveFileSizeForWrite_Async instead. ReserveFileSizeForWrite is the same as SetFileSize, but it uses some information about the platform and the file to decide whether the reserve will help or not. This function might do nothing if it thinks that the writes will be faster with no reservation.

The contents of the file in the resized but unwritten area are undefined/garbage.

See OodleXIOQ_SetFileSize_Async for more.
 

OodleXIOQ_SetFileSize_AsyncOodleX low level async ioOodleXIOQ_ForceWriteable_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLENETWORK1_MAX_DICTIONARY_SIZE
Navigation
 OODLENETWORK1_HASH_BITS_DEFAULT
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLENETWORK1_MAX_DICTIONARY_SIZE (1<<24)
Discussion
Maximum size of dictionary for OodleNetwork1
 
About OodleNetwork1OodleAPI_OodleNetwork1OODLENETWORK1_HASH_BITS_DEFAULT

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenForRead_Async
Navigation
 OodleXIOQ_Fence_Async
 OodleXIOQ_OpenAndRead_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenForRead_Async( OodleXIOQFile * pFile,
                                          const char * name,
                                          OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                          const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                          OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                          OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                          const OodleXHandle * dependencies OODEFAULT( NULL ),
                                          OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start opening a file for read
Parameters
pFile  filled with a handle to the file which will be opened
name  name of the file to open (VFS, UTF-8)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

If vtable is NULL, the vtable specified by the VFS mapping is used. The file name provided is automatically run through VFS-to-OS name mapping, if applicable.

Open returns a File ref right away for your convenience, but the file is not actually open for a little while. You can however go ahead and queue more requests on the file reference before open is complete. You cannot call things that require an open file, such as OodleXIOQ_GetInfo. OpenForRead is always shared access.

To also perform an initial read, use OodleXIOQ_OpenAndRead_Async


 

OodleXIOQ_Fence_AsyncOodleX low level async ioOodleXIOQ_OpenAndRead_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_UsesWholeBlockQuantum
Navigation
 OodleLZ_CompressOptions_Validate
 OodleLZ_Compressor_UsesLargeWindow
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_UsesWholeBlockQuantum( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor is "whole block quantum" ; must decode in steps of OODLELZ_BLOCK_LEN , not OODLELZ_QUANTUM_LEN like others.
 
OodleLZ_CompressOptions_ValidateOodleAPI_LZ_CompressorsOodleLZ_Compressor_UsesLargeWindow

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX Utils
Navigation
 OodleX Debug aids
 Oodle2 Ext API Documentation
 About Oodle Ext
 Welcome to Oodle
 Change Log
  •  Defines
    •  OODLEX_PATH_DELIM
  •  Functions
    •  OodleX_GetExtensionKey
    •  OodleX_MakeExtensionKey
    •  OodleX_IOAlignUpS32
    •  OodleX_IOAlignUpS64
    •  OodleX_IOAlignUpSINTa
    •  OodleX_IOAlignDownS32
    •  OodleX_IOAlignDownS64
    •  OodleX_IOAlignDownSINTa
    •  OodleX_S64_to_SINTa_check
    •  OodleX_GetSeconds
    •  OodleXUtil_ConvertUTF8ToUTF16
    •  OodleXUtil_ConvertUTF16ToUTF8
    •  OodleX_CombinePaths
    •  OodleX_GetOSCwd
    •  OodleX_PrefixOSCwd

 
OodleXLog_PrintfErrorOodle2 Ext API DocumentationOODLEX_PATH_DELIM

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_CorePlugin_WaitJob
Navigation
 OodleX_CorePlugin_RunJob
 OodleX_GetNumWorkerThreads
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_CorePlugin_WaitJob( OO_U64 job_handle,
                                void * user_ptr );
Discussion
Function to plug in the OodleX Worker system to OodleCore_Plugins_SetJobSystem
Discussion
NOTE : OodleX_Init does OodleCore_Plugins_SetJobSystem automatically.


 

OodleX_CorePlugin_RunJobOodleX threading utilOodleX_GetNumWorkerThreads

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMallocAligned
Navigation
 OodleXMalloc
 OodleXFree
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXMallocAligned( OO_SINTa bytes,
                            OO_S32 alignment );
Discussion
Allocate some memory with specified alignment
Parameters
bytes  the amount to allocate (must be > 0)
alignment  the desired alignment
Return Value
return  pointer to allocated memory
Discussion

alignment must be <= bytes. alignment must be power of 2. OodleXMalloc uses the installed OodleXMallocVTable. Pointer will be aligned to at least OODLE_MALLOC_MINIMUM_ALIGNMENT. If a malloc fails, any installed OodleXMallocFailedHandler will be called.
 

OodleXMallocOodleX Memory AllocatorsOodleXFree

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_SeekTable
Navigation
 OodleLZDecoder
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleLZ_SeekTable
{
    OodleLZ_Compressor  compressor;
    OO_BOOL             seekChunksIndependent;
    OO_S64              totalRawLen;
    OO_S64              totalCompLen;
    OO_S32              numSeekChunks;
    OO_S32              seekChunkLen;
    OO_U32 *            seekChunkCompLens;
    OO_U32 *            rawCRCs;
};
Discussion
A seek table, as created by OodleLZ_CreateSeekTable
Members
compressor  which compressor was used
seekChunksIndependent  are the seek chunks independent, or must they be decompressed in sequence
totalRawLen  total uncompressed data lenth
totalCompLen  sum of seekChunkCompLens
numSeekChunks  derived from rawLen & seekChunkLen
seekChunkLen  multiple of OODLELZ_BLOCK_LEN
seekChunkCompLens  array of compressed lengths of seek chunks
rawCRCs  crc of the raw bytes of the chunk (optional; NULL unless OodleLZSeekTable_Flags_MakeRawCRCs was specified)
Discussion
An OodleLZ_SeekTable can be created from any OodleLZ compressed data. It should be transmitted if you wish to do random-access or parallel decompression. One way to transmit it is to use an "OOZ" file which stores the OodleLZ_SeekTable in its header.
 
OodleLZDecoderOodleAPI_LZ_CompressorsOodleLZ_Compress

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc
Navigation
 OodleXMalloc_SetFailedHandler
 OodleXMallocAligned
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXMalloc( OO_SINTa bytes );
Discussion
Allocate some memory
Parameters
bytes  the amount to allocate (must be > 0)
Return Value
return  pointer to allocated memory
Discussion

OodleXMalloc uses the installed OodleXMallocVTable. Pointer will be aligned to at least OODLE_MALLOC_MINIMUM_ALIGNMENT. If a malloc fails, any installed OodleXMallocFailedHandler will be called.
 

OodleXMalloc_SetFailedHandlerOodleX Memory AllocatorsOodleXMallocAligned

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_CorePlugin_RunJob
Navigation
 OodleX_ReleaseThreadTLS
 OodleX_CorePlugin_WaitJob
 Welcome to Oodle
 Change Log
// Function prototype:
OO_U64 OodleX_CorePlugin_RunJob( t_fp_Oodle_Job * fp_job,
                                 void * job_data,
                                 OO_U64 * dependencies,
                                 int num_dependencies,
                                 void * user_ptr );
Discussion
Function to plug in the OodleX Worker system to OodleCore_Plugins_SetJobSystem
Discussion
NOTE : OodleX_Init does OodleCore_Plugins_SetJobSystem automatically.


 

OodleX_ReleaseThreadTLSOodleX threading utilOodleX_CorePlugin_WaitJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_SetConfigValues
Navigation
 OodleX_GetConfigValues
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_SetConfigValues( const OodleXConfigValues * ptr );
Discussion
Set OodleXConfigValues
Parameters
ptr  your desired OodleXConfigValues
Discussion

Sets the global OodleXConfigValues from your struct.

You should call OodleX_GetConfigValues to fill the struct, then change the values you want to change, then call OodleX_SetConfigValues.

This should generally be done before doing anything with Oodle (eg. even before OodleX_Init). Changing OodleXConfigValues after Oodle has started has undefined effects.
 

OodleX_GetConfigValuesOodleX Startup and ShutdownOodleX Memory Allocators

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ForceWriteable_Async
Navigation
 OodleXIOQ_ReserveFileSizeForWrite_Async
 OodleXIOQ_Delete_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_ForceWriteable_Async( const char * name,
                                             OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                             OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                             const OodleXHandle * dependencies OODEFAULT( NULL ),
                                             OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a force-writeable file request.
Parameters
name  the file to make writeable (VFS, UTF-8)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Asynchronously make a file writeable/deletable. Useful if the file might have read-only or other-user permissions and you want to modifity anyway.

A common use is to enqueue a OodleXIOQ_ForceWriteable_Async right before a DeleteFile or RenameFile.
 

OodleXIOQ_ReserveFileSizeForWrite_AsyncOodleX low level async ioOodleXIOQ_Delete_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_GetVerboseLevel
Navigation
 OodleXLog_GetCallback
 OodleXLog_SetVerboseLevel
 Welcome to Oodle
 Change Log
// Function prototype:
int OodleXLog_GetVerboseLevel( );
Discussion
Get the global verbose level
Return Value
return  the value
Discussion

See OodleXLog_SetVerboseLevel


 

OodleXLog_GetCallbackOodleX Debug aidsOodleXLog_SetVerboseLevel

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetInfoByName_Async
Navigation
 OodleXIOQ_FreeBufferIOAligned_Async
 OodleXIOQ_GetInfoByName_GetResult
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_GetInfoByName_Async( const char * name,
                                            OodleFileNotFoundIsAnError errorIfNotFound,
                                            OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                            OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                            const OodleXHandle * dependencies OODEFAULT( NULL ),
                                            OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an asynchronous GetInfo request
Parameters
name  the file name to query (VFS,UTF-8)
errorIfNotFound  (optional) should file-not-found be an error status or not? (see OodleFileNotFoundIsAnError)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Used to get an OodleXFileInfo without blocking the calling thread. Use OodleXIOQ_GetInfoByName_GetResult to get the result.
 

OodleXIOQ_FreeBufferIOAligned_AsyncOodleX low level async ioOodleXIOQ_GetInfoByName_GetResult

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_Free
Navigation
 t_fp_OodleCore_Plugin_MallocAligned
 t_fp_OodleCore_Plugin_RunJob
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleCore_Plugin_Free )( void * ptr );
Discussion
Function pointer type for OodleFree
Return Value
return  pointer to memory to free
Discussion


 

t_fp_OodleCore_Plugin_MallocAlignedCore pluginst_fp_OodleCore_Plugin_RunJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXUtil_ConvertUTF8ToUTF16
Navigation
 OodleX_GetSeconds
 OodleXUtil_ConvertUTF16ToUTF8
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXUtil_ConvertUTF8ToUTF16( const char * from,
                                       OO_U16 * to,
                                       int toSize );
Discussion
Convert a UTF8 string to UTF16
Parameters
from  string to convert
to  filled with the result
toSize  number of characters availabe in to (not bytes!)
Discussion

Fill [ to , to + toSize ] with the UTF16 conversion of from (UTF8).

Oodle strings are all UTF8. This can be used to convert a string to UTF16 , for example for use with the Windows wchar routines.

Note that OodleXUtil_ConvertUTF8ToUTF16 + OodleXUtil_ConvertUTF16ToUTF8 may not produce the same output, because UTF encodings are not unique. Also note that Windows 8-bit names (from "A" code page, though they are not ANSI or ASCII) are not UTF8.
 

OodleX_GetSecondsOodleX UtilsOodleXUtil_ConvertUTF16ToUTF8

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleEvent_SetDone
Navigation
 OodleXHandleEvent_Alloc
 OodleXHandleEvent_SetError
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXHandleEvent_SetDone( OodleXHandle h );
Discussion
Set an OodleXHandleEvent to OodleXStatus_Done
Parameters
h  handle created by OodleXHandleEvent_Alloc
Discussion

The state transition from Pending->Done is one way. If the handle is OodleXHandleAutoDelete_Yes, it goes away now.
 

OodleXHandleEvent_AllocOodleX async handle operationsOodleXHandleEvent_SetError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXAsyncSelect
Navigation
 OodleXPriority
 OodleXStatus
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXAsyncSelect
{
    OodleXAsyncSelect_None = 0,
    OodleXAsyncSelect_Workers = 0x100,
    OodleXAsyncSelect_NoFlagsMask = 0xFFF,
    OodleXAsyncSelect_Wide = 0x1000,
    OodleXAsyncSelect_Full = 0xFFFF,
    OodleXAsyncSelect_All = OodleXAsyncSelect_Full,
    OodleXAsyncSelect_Force32 = 0x40000000
};
Discussion
OodleXAsyncSelect are bit masks that can be combined to form an async selector.
Enumerants
OodleXAsyncSelect_None  run synchronously
OodleXAsyncSelect_Workers  run async on the Workers worker threads
OodleXAsyncSelect_NoFlagsMask  mask for all ways to run async ops
OodleXAsyncSelect_Wide  flag : run async wide, use all possible runners
OodleXAsyncSelect_Full  full speed : just turn on all bits
OodleXAsyncSelect_All  synonym
OodleXAsyncSelect_Force32 
Discussion
The async selector tells an async operation like OodleXLZ_Decompress_Narrow_Async where it should run its decompress.

OodleXAsyncSelect_Wide means break the task into many smaller pieces that can be run simultaneously, and consume all available runners to make the task complete as quickly as possible. If WIDE is not specified, then the default is "narrow", that is run async but don't split the task for minimum latency. Mainly used with OodleXAsyncSelect_Workers ; WIDE means create several smaller Worklet, while narrow creates just one Worklet that does the whole task.

OodleXAsyncSelect_Full provides the quickest completion of any one call, but perhaps more contention with other operations.
 

OodleXPriorityOodleX async handle operationsOodleXStatus

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
Navigation
 example_lz_noallocs : Example demonstrating Oodle compression with no allocations
 Examples
 example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz_overlap

Demonstration of the benefit of overlapping IO with CPU work and parallelism in LZ decompression.

This example compresses a file, then repeatedly reads the compressed data and decompresses it, in several different ways.

There are two types of parallelism demonstrated here :

1. IO overlap. When reading and decompressing large files, you get minimum latency by overlapping the IO with the decompress. This is done by reading the compressed data in smaller chunks and decompressing each chunk (in parallel) as it is done.

2. Parallel "wide" decompression. OodleLZ with seek chunk resets can decompress using many threads simultaneously.

Combining IO overlap and wide decompression is the fastest way to load compressed data.


#include "../include/oodle2x.h"
#include "ooex.h"

#ifdef _MSC_VER
#pragma warning(disable : 4127) // conditional is constant
#endif

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#include "make_example_input.h"

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz_overlap
#endif

extern "C" int main(int argc,char *argv[])
{
    if ( ! OodleX_Init_Default(OODLE_HEADER_VERSION) )
    //if ( ! OodleX_Init_Default(OODLE_HEADER_VERSION,OodleX_Init_GetDefaults_DebugSystems_Yes,OodleX_Init_GetDefaults_Threads_No) )
    {
        fprintf(stderr,"OodleX_Init failed.\n");
        return 10;
    }

Pass in the file name to compress. If you don't provide one, I'll make one.

Ideally give this example something big to work on, 20M - 100M , to get better charts.


    // get args :
    const char * inName;
    if ( argc < 2 )
    {
        inName = "oodle_example_input_file";
        make_example_input(inName);
    }
    else
    {
        inName = argv[1];
    }

    // we'll write to this file :
    const char * compName = "oodle_example_overlap_comp";
     
    OodleXLog_Printf_v1("compressing %s -> %s ...\n",inName,compName);

Set up the LZ compression options so that we make independent chunks (seekChunkReset = true) and make them 1 MB each.

This hurts compression ratio (vs not chunking) but allows decompression of each chunk to run in parallel.

    
    OodleLZ_CompressionLevel lzLevel = OodleLZ_CompressionLevel_Fast;
    OodleLZ_Compressor lzCompressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressOptions lzOptions = * OodleLZ_CompressOptions_GetDefault(lzCompressor,lzLevel);

    lzOptions.seekChunkLen = 1<<20; // 1 MB
    lzOptions.seekChunkReset = true;

Read the input file, compress it, and write it out :

In practice for maximum speed, you could read each chunk of the input and compress them independently as each chunk async IO read finishes.


    OO_SINTa inSize;
    void * inBuf;
    OO_SINTa compSize;
    OodleLZ_SeekTable * seekTable;
    OO_S64 inSize64;
    
    inBuf = OodleXIOQ_ReadMallocWholeFile_AsyncAndWait(inName,&inSize64);
    if ( inBuf == NULL )
    {
        OodleXLog_Printf_v0("ERROR couldn't load : %s\n",inName);
        return 10;
    }

    inSize = OodleX_S64_to_SINTa_check(inSize64);

    { // scope for compBuf
        void * compBuf = OodleXMalloc_IOAligned( OodleLZ_GetCompressedBufferSizeNeeded(lzCompressor,inSize) );
        
        OO_U32 asyncSelect = OodleXAsyncSelect_All;
        compSize = OodleXLZ_Compress_AsyncAndWait(asyncSelect,lzCompressor,inBuf,inSize,compBuf,lzLevel,&lzOptions);
        
        if ( compSize <= 0 )
        {
            OodleXLog_Printf_v0("ERROR failed to compress\n");
            return 10;
        }
        
        if ( ! OodleXIOQ_WriteWholeFile_AsyncAndWait(compName,compBuf,compSize) )
        {
            OodleXLog_Printf_v0("ERROR couldn't write : %s\n",compName);
            return 10;
        }
        
        // log about it :
        OodleXLog_Printf_v1("%s compressed %d -> %d\n",inName,(int)inSize,(int)compSize);

    
Make an OodleLZ_SeekTable on the compressed data for later use with parallel decompression.

You should store the seektable to disk with any header information, and load it before loading the compressed bulk data. Oodle can do this for you with an "OOZ" file.

For this example we'll just keep the seekTable in memory. In real use you would write the seek table with the compressed data.

Once we make the seekTable, we free compBuf - to get the compressed data again we will have to read it from disk.

        
        seekTable = OodleLZ_CreateSeekTable(OodleLZSeekTable_Flags_None,
                                                        lzOptions.seekChunkLen,NULL,inSize,compBuf,compSize);
        
        OodleXFree_IOAligned(compBuf);
        compBuf = NULL;
    }
    
    // we'll decompress into decompBuf
    void * decompBuf = OodleXMalloc(inSize);
    
    //===================================================================================

    // prevent cheating :
    memset(decompBuf,(int)rand(),inSize);

Read the files with unbuffered IO

We're mainly doing this here for consistent benchmarking so we can see IO times otherwise the files would just come from the OS buffers and no show real IO time. In practice you should usually use buffered IO. (OodleXFileOpenFlags_Default)


    OodleXFileOpenFlags fileOpenFlags = OodleXFileOpenFlags_NotBuffered;

Now begins various ways to read the compressed file and decompress it.

Read-Decomp 1 :

Read the whole compressed file, synchronously. When that's done, decompress the whole buffer, synchronously.

Simple, but stalls the main thread and gets no overlap of IO or parallelism in the decompress.

    
    OodleXLog_Printf_v1("Doing read then decomp, synchronously on the main thread :\n");
    
    {
    void * compBuf = OodleXIOQ_ReadMallocWholeFile_AsyncAndWait(compName,NULL,fileOpenFlags);
    if ( compBuf == NULL )
    {
        OodleXLog_Printf_v0("ReadMallocWholeFile failed on %s\n",compName);
        return 10;
    }
    
    OodleLZ_Decompress(compBuf,compSize,decompBuf,inSize,OodleLZ_FuzzSafe_Yes);
    
    // check it :
    OOEX_ASSERT( memcmp(inBuf,decompBuf,inSize) == 0 );
    
    OodleXFree_IOAligned(compBuf);
    }
    
    //===================================================================================
    
Read-Decomp 2 :

Read the whole compressed file, and fire off a full-buffer decompress, scheduled to run automatically when the read is done.

This has the same latency as method 1, but is a single async operation so the main thread can do something else the whole time.


    OodleXLog_Printf_v1("Doing read then decomp, through a job chain :\n");
    
    // prevent cheating :
    memset(decompBuf,(int)rand(),inSize);
    
    {
    
    // for simplicity, we'll just malloc compBuf with our known compSize
    // more generally if you didn't know compSize, you would have to use an OodleWork coroutine
    //  to first open the file, get the size, do the malloc, do the read, then the decompress
    void * compBuf = OodleXMalloc_IOAligned(compSize);

    // make an IO request to open and read the whole file : 
    OodleXIOQFile compFile;
    OodleXHandle openAndReadH = OodleXIOQ_OpenAndRead_Async(&compFile,compName,compBuf,OodleX_IOAlignUpSINTa(compSize),0,fileOpenFlags,0,OodleXHandleAutoDelete_Yes);

    // go ahead and enqueue a Close to follow the OpenAndRead :
    OodleXIOQ_CloseFile_Async(compFile,OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE,OodleXHandleAutoDelete_Yes);

    // the decompress depends on the OpenAndRead - it will run only when that is done
    //  dependencies are an array, but we have only one, so just point at it :
    const OodleXHandle * deps = &openAndReadH;
    int num_deps = 1;
    
    OodleXHandle decompH = OodleXLZ_Decompress_Narrow_Async(OodleXAsyncSelect_Full,compBuf,compSize,decompBuf,inSize,
                                        OodleLZ_FuzzSafe_No,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,
                                        0,0,0,0,0,0,OodleLZ_Decode_Unthreaded,0,0,0,0,
                                        OodleXHandleAutoDelete_No,
                                        deps,num_deps);
    
    // ... main thread can do other work here ...
    
    OodleXStatus st = OodleX_WaitAndDelete(decompH);
    if ( st != OodleXStatus_Done )
    {
        OodleXLog_Printf_v0("OodleXLZ_Decompress_Narrow_Async failed!\n");
        return 10;
    }
    
    // check it :
    OOEX_ASSERT( memcmp(inBuf,decompBuf,inSize) == 0 );
    
    OodleXFree_IOAligned(compBuf);
    }
    
    //===================================================================================
    
Read-Decomp 3 :

Read and decompress with IO overlap, but only using a single thread (not "wide").

OodleXLZ_ReadAndDecompress_Stream_Async is an API provided to do IO overlap with decompression. It's something you could easily write yourself in Oodle. It uses a coroutine to do IO on chunks and then decompress the chunks as they arive. It tries to always be doing the IO for the next chunk while decompressing the current chunk.

    
    OodleXLog_Printf_v1("Doing read and decomp simultaneously :\n");
    
    // prevent cheating :
    memset(decompBuf,(int)rand(),inSize);
    
    {
    void * compBuf = OodleXMalloc_IOAligned(compSize);

    
Enqueue an open request + read an initial small chunk into compBuf

We don't wait on the Open, but immediately start the OodleXLZ_ReadAndDecompress_Stream_Async operation, and pass it the openAndRead handle.

  
    
    OodleXIOQFile compFile;
    OO_SINTa initialReadSize = OodleX_IOAlignUpSINTa( OOEX_MIN(compSize,512*1024) );
    OodleXHandle openAndReadH = OodleXIOQ_OpenAndRead_Async(&compFile,compName,compBuf,initialReadSize,0,fileOpenFlags);
    
    OodleXHandle readAndDecomp = OodleXLZ_ReadAndDecompress_Stream_Async(OodleXAsyncSelect_Full,
                                            compBuf,compSize,decompBuf,inSize,
                                            OodleLZ_FuzzSafe_No,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,0,0,
                                            compFile,compBuf,0,openAndReadH,initialReadSize);
                                            
    // enqueue a CloseFile with a dependency on the ReadAndDecomp operation :
    OodleXIOQ_CloseFile_Async(compFile,OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE,OodleXHandleAutoDelete_Yes,OodleXPriority_Normal,&readAndDecomp,1);
    
    // ... main thread can do other work ...
    
    OodleXStatus st = OodleX_WaitAndDelete(readAndDecomp);
    if ( st != OodleXStatus_Done )
    {
        OodleXLog_Printf_v0("ReadAndDecomp failed\n");
        return 10;
    }
    
    // check it :
    OOEX_ASSERT( memcmp(inBuf,decompBuf,inSize) == 0 );
    
    OodleXFree_IOAligned(compBuf);
    }
    
    //===================================================================================
        
Read-Decomp 4 :

Read and decompress with IO overlap, and using all worker threads ("wide").

ReadAndDecompress_Wide needs the seekTable to find the compressed block boundaries. Normally you would have to send that in a file (see the OOZ APIs if you want Oodle to do it for you). Here we just use the seekTable we made earlier and kept in memory.


    OodleXLog_Printf_v1("Doing read and decomp wide :\n");
    
    // prevent cheating :
    memset(decompBuf,(int)rand(),inSize);
    
    {
    void * compBuf = OodleXMalloc_IOAligned(compSize);
    
    // open compressed file and do an initial read :
    OodleXIOQFile compFile;
    OO_SINTa initialReadSize = OodleX_IOAlignUpSINTa( OOEX_MIN(compSize,256*1024) );
    OodleXHandle openAndReadH = OodleXIOQ_OpenAndRead_Async(&compFile,compName,compBuf,initialReadSize,0,fileOpenFlags);

    // instead of waiting on openAndReadH here, you could pass it as a dependency
    //  to the OodleXLZ_ReadAndDecompress_Wide_Async function to make the whole sequence async
    // but we'll just stall here for simplicity in this example :
    OodleXStatus st = OodleX_WaitAndDelete(openAndReadH);
    
    if ( st != OodleXStatus_Done )
    {
        OodleXLog_Printf_v0("OpenAndRead failed\n");
        return 10;
    }
    
    // fire off the read-and-decomp job :   
    OodleXHandle readAndDecomp = OodleXLZ_ReadAndDecompress_Wide_Async(OodleXAsyncSelect_Full,
                                            seekTable,compBuf,compSize,initialReadSize,compFile,0,
                                            decompBuf,inSize,
                                            OodleLZ_FuzzSafe_No,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None);

    // enqueue a CloseFile with a dependency on the ReadAndDecomp operation :
    OodleXIOQ_CloseFile_Async(compFile,OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE,OodleXHandleAutoDelete_Yes,OodleXPriority_Normal,&readAndDecomp,1);

    // ... main thread can do other work ...
        
    st = OodleX_WaitAndDelete(readAndDecomp);
    
    if ( st != OodleXStatus_Done )
    {
        OodleXLog_Printf_v0("ReadAndDecomp failed\n");
        return 10;
    }
    
    // check it :
    OOEX_ASSERT( memcmp(inBuf,decompBuf,inSize) == 0 );
    
    OodleXFree_IOAligned(compBuf);
    }
    
    //===================================================================================
    // all done, clean up :
    
    OodleLZ_FreeSeekTable(seekTable);
    
    OodleXFree(decompBuf);
    OodleXFree_IOAligned(inBuf);

    OodleX_Shutdown();

    return 0;
}



 
example_lz_noallocs : Example demonstrating Oodle compression with no allocationsExamplesexample_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetConfigValues
Navigation
 OodleX_Shutdown_NoThreads
 OodleX_SetConfigValues
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_GetConfigValues( OodleXConfigValues * ptr );
Discussion
Get OodleXConfigValues
Parameters
ptr  filled with OodleXConfigValues
Discussion

Gets the current OodleXConfigValues.

May be different per platform.
 

OodleX_Shutdown_NoThreadsOodleX Startup and ShutdownOodleX_SetConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_GetVTable_OS
Navigation
 OodleXMalloc_GetVTable_Clib
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleXMallocVTable * OodleXMalloc_GetVTable_OS( OodleXMalloc_OS_Options options );
Discussion
get an OodleXMalloc VTable that contains allocators based on the OS system allocators
Parameters
options  see OodleXMalloc_OS_Options; default is OodleXMalloc_OS_Options_None
Discussion

This is usually used to set OodleXInitOptions:m_pBaseVTable
 

OodleXMalloc_GetVTable_ClibOodleX Memory AllocatorsOodleXMallocFailedHandler

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleDecompressCallback_WriteFile
Navigation
 OodleXDecompressCallback_WriteFile_Data_Init
 Welcome to Oodle
 Change Log
// Function prototype:
OodleDecompressCallbackRet OODLE_CALLBACK OodleDecompressCallback_WriteFile( void * pcbData,
                                                                             const OO_U8 * rawBuf,
                                                                             OO_SINTa rawLen,
                                                                             const OO_U8 * compBuf,
                                                                             OO_SINTa compBufferSize,
                                                                             OO_SINTa rawDone,
                                                                             OO_SINTa compUsed );
Discussion
A OodleDecompressCallback which writes the decompressed data to a file
Parameters
pcbData  the OodleDecompressCallback_WriteFile_Data you passed to OodleLZ_Decompress
rawBuf  the decompressed buffer
rawLen  the total decompressed length
compBuf  the compressed buffer
compBufferSize  the total compressed length
rawDone  number of bytes in rawBuf decompressed so far
compUsed  number of bytes in compBuf consumed so far
Discussion

OodleDecompressCallback is called incrementally during decompression.

This is provided as a convenience for use as an OodleDecompressCallback in functions that take that callback, such as OodleXLZ_ReadAndDecompress_Stream_Async.

NOTE : you typically need to do OodleX_WaitAndDelete on the closeH from OodleDecompressCallback_WriteFile_Data
 

OodleXDecompressCallback_WriteFile_Data_InitOodleXAPI_LZ_AsyncOodleX Startup and Shutdown

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_MakeSeekChunkLen
Navigation
 OodleLZDecoder_MakeValidCircularWindowSize
 OodleLZ_GetNumSeekChunks
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZ_MakeSeekChunkLen( OO_S64 rawLen,
                                 OO_S32 desiredSeekPointCount );
Discussion
Compute a valid seekChunkLen
Parameters
rawLen  total length of uncompressed data
desiredSeekPointCount  desired number of seek chunks
Return Value
return  a valid seekChunkLen for use in OodleLZ_CreateSeekTable
Discussion

Returns a seekChunkLen which is close to (rawLen/desiredSeekPointCount) but is a power of two multiple of OODLELZ_BLOCK_LEN

desiredSeekPointCount = 16 is good for parallel decompression. (OODLELZ_SEEKPOINTCOUNT_DEFAULT)
 

OodleLZDecoder_MakeValidCircularWindowSizeOodleAPI_LZ_CompressorsOodleLZ_GetNumSeekChunks

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_Shared_Size
Navigation
 OodleNetwork1TCP_State_Size
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1_Shared_Size( OO_S32 htbits );
Discussion
Returns the size of memory required for an OodleNetwork1_Shared object
Parameters
htbits  size of the OodleNetwork1 hash table (log2) ; typically 18-21 such as OODLENETWORK1_HASH_BITS_DEFAULT
Discussion

Shared and State are allocated with malloc( Size() )


 

OodleNetwork1UDP_StateCompactedOodleAPI_OodleNetwork1OodleNetwork1TCP_State_Size

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Third Party License Notices
Navigation
 Acknowledgements
 Welcome to Oodle
 Change Log

Oodle uses "ConvertUTF" by Unicode, Inc. License notice follows :


 * Copyright 2001-2004 Unicode, Inc.
 * 
 * Disclaimer
 * 
 * This source code is provided as is by Unicode, Inc. No claims are
 * made as to fitness for any particular purpose. No warranties of any
 * kind are expressed or implied. The recipient agrees to determine
 * applicability of information provided. If this file has been
 * purchased on magnetic or optical media from Unicode, Inc., the
 * sole remedy for any claim will be exchange of defective media
 * within 90 days of receipt.
 * 
 * Limitations on Rights to Redistribute This Code
 * 
 * Unicode, Inc. hereby grants the right to freely use the information
 * supplied in this file in the creation of products supporting the
 * Unicode Standard, and to make copies of this file in any form
 * for internal or external distribution as long as this notice
 * remains attached.


The Oodle examples are shipped with some code from cbloom.com ; the code is Public Domain.



 
AcknowledgementsAcknowledgements 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleXHandle
Navigation
 OodleX async handle operations
 Welcome to Oodle
 Change Log

About OodleXHandle

OodleXHandle is the generic handle to an async operation. Every async operation performed by Oodle corresponds to an OodleXHandle.

(See also About Oodle Ext)

OodleXHandle is a user-mode waitable handle. It is very light weight; it provides lock-free status and liveness checks. It provides real waits without polling or "thread thrashing" or timed Sleeps, all of which hurt thread performance greatly.

An OodleXHandle is a "weak reference". That means that the object it refers to can be deleted at any time, and the handle stays a valid unique reference to that object. If you try to do an operation (like OodleX_GetStatus) on an object that was deleted, it will return OodleXStatus_Invalid.

In general, Oodle considers a handle that doesn't exist to be equivalent to a handle that's Done. eg. if you start an operation with a dependency on a handle that doesn't exist, the operation can start immediately.

All operations on OodleXHandle are thread safe and atomic. eg. if two threads try to delete a handle at the same time, only one of them will get the delete, and the other will either see the handle before the deletion or after the deletion (not during).

You can always use OodleX_Wait or OodleX_GetStatus on any handle. You can combine handles from various subsystems and OodleX_WaitAll on all of them. There are also subsystem-specific functions, and it's up to you to use them correctly. For example if you call OodleAsyncGroup_Lock on a handle that is not an OodleAsyncGroup, it will fail in an undefined way.

OodleXHandles always have a strict progression through their autoDelete. When they are first created they are OodleXStatus_Pending. They stay Pending while they are being run, and then they are set to OodleXStatus_Done or OodleXStatus_Error. (to check completion you should not use == OodleXStatus_Done, instead use != OodleXStatus_Pending). A call to OodleX_Wait on a handle will return when status is not Pending. Then a handle is deleted, and any further checks on it return OodleXStatus_Invalid. A single handle can never go from Done back to Pending - if there is more work to do, you must create a new handle for the next bit of work.

OodleXHandles with a autoDelete of OodleXHandleAutoDelete_Yes may never actually be set to OodleXStatus_Done, they go directly from OodleXStatus_Pending to deleted. The state transition from pending to deleted still triggers any waits on that handle.

Oodle does not delete handles for you (unless you tell it to). OodleXHandle lifetimes are managed by the client. When you fire off an async operation, you often get the option of whether the handle should be "autoDelete" or not. If a handle is autoDelete, as soon as its status becomes not-Pending (Done or Error), the handle is deleted by the system. A deleted handle is "not Pending" so it satisfies OodleX_Wait, but you can't check if it failed and you can't get any results from the operation. AutoDelete operations are "fire and forget"; they are convenient because you don't have to free the handle, but can be hard to use safely.

The other option for handle lifetimes is OodleXHandleAutoDelete_No (the default) ; if a handle is not AutoDelete, then at some point you must free it. This allows you to get results and check for errors before it is deleted. The most common way to delete a handle is via OodleX_Wait with OodleXHandleDeleteIfDone_Yes. The other standard way is via handle-specific completion functions, such as Oodle_CopyFile_Wait_GetResult.

Deleting handles while they are still pending is undefined in general; it is possible in some specific situations. (it can be done by calling a Cancel function)

If handles are not deleted, they will eventually take all the slots in the handle table. Handle leaks can be found with OodleLeakTrack .

All OodleXHandles must be able to fit in a static table that is created by OodleInit. The size is determined by OodleXInitOptions:m_num_handles_log2 (times 2048 since Oodle 2.9.7). If you run out of handles during execution, Oodle will fail catastrophically. See FAQ: I ran out of OodleHandle table slots; what do I do?.
 

OodleX async handle operationsOodleX async handle operationsOODLEX_ASYNC_HANDLE_INVALID

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Windows
Navigation
 About Oodle on Platforms
 About Oodle on PS4
 Welcome to Oodle
 Change Log

This section applies to Win32 Platforms. See also About Oodle on Windows UWP for UWP / WinRT / Windows App / Phone platforms.

Oodle on Windows is now built with MSVC 2017. It is compatible with MSVC 2015+.

Oodle on Windows is shipped as a DLL. This allows it to avoid linker conflicts with your application, primarily caused by choice of C runtime. Each major rev of Oodle has a new DLL name, so there is no possibility of importing a DLL version that doesn't match what you app was built with. To ensure the DLL version matches your app, OodleX_Init or Oodle_CheckVersion check the version number and will fail if they are incompatible.

To build with the Oodle DLL on Windows you link your app with just one of these libs :

lib/oo2core_win32.lib
lib/oo2core_win64.lib
lib/oo2ext_win32.lib
lib/oo2ext_win64.lib

and ship your app with one of these dll's :

redist/oo2core_##_win32.dll
redist/oo2ext_##_win32.dll
redist/oo2core_##_win64.dll
redist/oo2ext_##_win64.dll

Where ## is the major version number of Oodle. The libs are import libs that you link with your game; they know the name of the appropriate DLL to load. The DLLs in the redist directory may be shipped with your game.

The debug build of the Oodle DLL is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems. Debug versions of the DLLs are provided in the "redistdebug" directory. These should not be shipped with your game.


static libs on Windows are now provided as an option. These may be incompatible with your build due to choice of compiler flags. If the static libs are incompatible, please use the DLL option to resolve the problem. Unfortunately with static libs it is impossible to ensure broad link time compatibility.

To use the static libs, link with a lib from the "static_lib/" directory.


NOTE! WARNING! IMPORTANT! For users of OodleX DLL :

On Windows, the oo2ext DLL contains both OodleX and Oodle Core. Don't link with both oo2core and oo2ext, just one or the other.

(on all other platforms, oo2ext is an addition to oo2core and you need both if you use OodleX)

DO NOT link both core and ext on Windows - if you like Ext it includes a copy of Core. Additionally linking Core is NOT benign, it will cause Ext to fail to set up the Core plugins consistently.


To build an Oodle Example :

  • use MSVC 2015+
  • make a new console project for x64, make it empty
  • add an example cpp file
  • add the oodle release 64 bit lib , either from lib/ or static_lib/
  • if you use the DLL import lib, change your run working directory to the redist dir


Oodle assumes that Windows machines have SSE2. Newer processor features are detected dynamically.


The OodleX log file is written to c:\oodlelogs on Windows unless you have set m_OodleInit_Log_FileName.

NOTE : if you are using Core only on a Microsoft target, note that the default core log function only goes to OutputDebugString - not stdio - so you may not see any error messages unless you are running in the debugger. Either use Oodle X or install your own logger to get messages however you want.
 

About Oodle on PlatformsAbout Oodle on PlatformsAbout Oodle on Windows UWP

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How much memory do the Oodle compressors use ?
Navigation
 FAQ: How do I limit the encoder memory use?
 Frequently Asked Questions
 FAQ: What are the Oodle deprecated compressors ?
 Welcome to Oodle
 Change Log

The Oodle decoders need an input buffer and an output buffer which you provide.

In addition to that, they need an OodleLZDecoder object which is a fixed size. The size is reported by OodleLZDecoder_MemorySizeNeeded so you can allocate the memory yourself if you like (or put it on the stack), and Oodle decoding will then do no allocations. When you call OodleLZ_Decompress without provided memory, this allocation is done for you.

(The input & output buffer can be circular to reduce memory use if you are streaming a file through the decoder without ever holding the whole compressed or decompressed file in memory, but that's not recommended for normal use as it reduces performance)

The amount of memory needed to decode each OodleLZ_Compressor is different.

For Kraken, Mermaid, Selkie & Leviathan the current (2.7) OodleLZDecoder_MemorySizeNeeded is 442552 bytes (but don't hard code that value, it could change!)

This Decoder object scratch is used during the decode call, but not retained. So you can reuse the same memory for an unrelated (but not simultaneous on another thread) decode.

The decoder object memory can be passed in to OodleLZ_Decompress using the decoderMemory and decoderMemorySize arguments. If you pass in this memory, OodleLZ_Decompress will do no allocations.

The memory use needed for encoding is a different story.

The memory use for encoding generally goes up at higher compression levels. It can be quite small at levels Fast and VeryFast, and can be quite high at levels Optimal2 and higher. It's always recommended to run the Optimal encoders in 64-bit on machines with plenty of RAM (at least 4 GB and ideally 8 GB).

The encoders now accept passed-in memory which they will use before trying to allocate. If they run out of the scratch you passed in, they will call the installed Malloc and expect to get memory back from it. They do not handle Malloc failure gracefully. You should always provide a plugin allocator even if you expect it not to be used.

See FAQ: How do I limit the encoder memory use?
 

FAQ: How do I limit the encoder memory use?Frequently Asked QuestionsFAQ: What are the Oodle deprecated compressors ?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_ASYNC_HANDLE_PENDING
Navigation
 OODLEX_ASYNC_HANDLE_INVALID
 OODLEX_ASYNC_HANDLE_DONE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_ASYNC_HANDLE_PENDING ((OodleXHandle)0x0000000000000001ULL)
Discussion
OodleXHandle to a special always-pending handle. This is for Oodle internal use only. Calls to OodleX_GetStatus on this handle value will return &OodleXStatus_Pending. This is designed for use with OodleAsyncGroup. See OodleAsyncGroup_ChangePending. Calling OodleX_Wait on this handle is a deadlock. This handle must not be deleted! Do not call OodleX_Wait on it with deleteIfDone = true.
 
OODLEX_ASYNC_HANDLE_INVALIDOodleX async handle operationsOODLEX_ASYNC_HANDLE_DONE

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleXMalloc
Navigation
 OodleX Memory Allocators
 Welcome to Oodle
 Change Log

About OodleXMalloc

OodleXMalloc is the allocator used internally by OodleX to allocate memory.

NOTE : if you use Oodle Ext , OodleXMalloc must be initialized for OodleX internal use. If you use Oodle Core, you do not need to use OodleXMalloc, you may plug in your own malloc. You cannot use your own malloc for Oodle Core if you are using OodleX - they cannot use different allocators. See Core plugins if you only need to change the Oodle Core malloc.

You may wish to point OodleX at your own allocator. This is done by installing an OodleXMallocVTable in OodleX_Init.

The OodleX allocator provides an implementation for the malloc needed by Oodle Core. It also adds functionality for doing IO-aligned allocations (required for async IO on some platforms).

Most of the low-level Oodle Core runtime calls (like OodleLZDecoder_Create) can be optionally passed in pre-allocated memory so that they don't do any allocations at all.

Sometimes OodleX does an allocation for you and passes back the pointer, in which case you must free it with OodleXFree.

There are two types of memory used in OodleX :

  • Normal general purpose memory, as allocated by OodleXMalloc / OodleXFree

  • IO-aligned memory, as allocated by OodleXMalloc_IOAligned / OodleXFree_IOAligned ; this is just system memory, but aligned to OODLEX_IO_MAX_ALIGNMENT

The low level async IO systems (primarily OodleXIOQ ; see About OodleIOQ ) require IO-aligned memory. You do not necessarily have to use OodleXMalloc_IOAligned to get this memory, you can use your own malloc, but then you must ensure OODLEX_IO_MAX_ALIGNMENT. If you ask OodleX to do the allocation for you (eg. by calling something like OodleXIOQ_OpenAndReadMallocWholeFile_Async), then you must free the pointer with OodleXFree_IOAligned.

If you don't provide an allocator, OodleX will install its own, which calls through to the low level system allocator.

OodleX expects malloc to never fail. It doesn't necessarily return clean errors if it runs out of memory deep in a function. If the installed OodleXMalloc returns NULL, then OodleX calls the OodleXMallocFailedHandler that's set by OodleXMalloc_SetFailedHandler. In the FailedHandler you could abort your game, or try to free up memory, and then return true, in which case OodleX will try the allocation again.

For clients interested in replacing OodleXMalloc with their own implementation, read on.

All OodleX allocations go through an OodleXMallocVTable. To install your own allocator, you must implement the functions needed and fill out an OodleXMallocVTable with pointers to your functions. Then set OodleXInitOptions:m_pBaseVTable to your own VTable before calling OodleX_Init. Generally you don't want to change the installed Malloc VTable between the calls to OodleX_Init and OodleX_Shutdown - you should install one in Init and not change it.

By default, OodleXInitOptions has m_OodleInit_LockFreePageAllocator set to true; that puts the lock free page allocator on top of your allocator. If you don't want that, then set it to false.

OodleXMalloc works by calling OodleXMallocCall on the global installed VTable. OodleXMallocCall can be used to invoke a malloc from any vtable.

See OodleXMallocVTable for notes on how each function you provide must behave.

The functions you provide must all be thread safe.

One note on m_pMallocBig : you have choice about how to implement this. You can just do a malloc that provides memory aligned to OODLEX_IO_MAX_ALIGNMENT. If you do that, then OodleXMalloc_IOAligned will simply call directly to your m_pMallocBig. Alternatively, m_pMallocBig can be the system large page alloc (eg. on Windows it would be VirtualAlloc). If you do that, then OodleXMalloc_IOAligned will allocate from portions of the "Big" pages.

The default OodleXMalloc sets m_pBaseVTable to the OS allocators (not clib's malloc/free, but the lower level allocators; for example on Windows it uses HeapAlloc for m_pMalloc and VirtualAlloc for m_pMallocBig). The default then also has m_OodleInit_LockFreePageAllocator set to true, which creates a layered OodleXMallocVTable which uses its own lock-free page allocator for small allocs, and passes through large allocs to the base VTable (the OS allocators).
 

OodleX Memory AllocatorsOodleX Memory AllocatorsOodleXMalloc_OS_Options

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFileInfo
Navigation
 OodleXFileOpsVTable
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleXFileInfo
{
    OO_U32  flags;
    OO_U32  pad;
    OO_S64  size;
    OO_U64  modTime;
};
Members
flags  logical or of OODLEX_FILEINFO_FLAGS
size  file size; OODLEX_FILE_SIZE_INVALID if unknown
modTime  modTime on different platforms doesn't necessarilly mean anything, but it should be comparable with integer < and == (on the same platform, not vs. other platforms)

 
OodleXFileOpsVTableOodleX low level async ioOodleXIOQ_WaitDoneAllPending

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILEINFO_FLAG_INVALID
Navigation
 OODLEX_BUFFER_SIZE_DEFAULT
 OODLEX_FILEINFO_MODTIME_INVALID
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_FILEINFO_FLAG_INVALID ((OO_U32)-1)
Discussion
Invalid value for OodleXFileInfo:flags
 
OODLEX_BUFFER_SIZE_DEFAULTOodleX low level async ioOODLEX_FILEINFO_MODTIME_INVALID

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State_Uncompact_ForVersion
Navigation
 OodleNetwork1UDP_State_Compact
 OodleNetwork1UDP_State_Uncompact
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1UDP_State_Uncompact_ForVersion( OodleNetwork1UDP_State * to,
                                                     const OodleNetwork1UDP_StateCompacted * from,
                                                     OO_S32 for_oodle_major_version );
Discussion
See OodleNetwork1UDP_State_Uncompact * * takes oodle_major_version to target * * Oodle Network Compacted state changed from major version 5 to 6 (eg 2.5.5 to 2.6.0) *
 
OodleNetwork1UDP_State_CompactOodleAPI_OodleNetwork1OodleNetwork1UDP_State_Uncompact

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_BLOCK_MAX_COMPLEN
Navigation
 OODLELZ_BLOCK_LEN
 OODLELZ_QUANTUM_LEN
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_BLOCK_MAX_COMPLEN (OODLELZ_BLOCK_LEN+OODLELZ_BLOCK_MAXIMUM_EXPANSION)
Discussion
Maximum expansion per OODLELZ_BLOCK_LEN is 1 byte. Note that the compressed buffer must be allocated bigger than this (use OodleLZ_GetCompressedBufferSizeNeeded)
 
OODLELZ_BLOCK_LENOodleAPI_LZ_CompressorsOODLELZ_QUANTUM_LEN

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_MakeExtensionKey
Navigation
 OodleX_GetExtensionKey
 OodleX_IOAlignUpS32
 Welcome to Oodle
 Change Log
// Function prototype:
OO_U32 OodleX_MakeExtensionKey( const char * extension );
Discussion
convert the file name extension (max 4 chars) into a U32 for fast compares
Parameters
extension  string of an extension, not including period
Return Value
return  a U32 with up to 4 chars of extension
Discussion

The return value is case-insensitive and does not include the dot.
 

OodleX_GetExtensionKeyOodleX UtilsOodleX_IOAlignUpS32

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetCompressedStepForRawStep
Navigation
 OodleLZ_GetInPlaceDecodeBufferSize
 OodleLZ_GetAllChunksCompressor
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetCompressedStepForRawStep( const void * compPtr,
                                              OO_SINTa compAvail,
                                              OO_SINTa startRawPos,
                                              OO_SINTa rawSeekBytes,
                                              OO_SINTa * pEndRawPos OODEFAULT( NULL ),
                                              OO_BOOL * pIndependent OODEFAULT( NULL ) );
Discussion
How many bytes to step a compressed pointer to advance a certain uncompressed amount
Parameters
compPtr  current compressed pointer
compAvail  compressed bytes available at compPtr
startRawPos  initial raw pos (corresponding to compPtr)
rawSeekBytes  the desired step in raw bytes, must be a multiple of OODLELZ_QUANTUM_LEN or OODLELZ_BLOCK_LEN
pEndRawPos  (optional) filled with the end raw pos actually reached
pIndependent  (optional) filled with a bool that is true if the current chunk is independent from previous
Return Value
return  the number of compressed bytes to step
Discussion

You should try to use GetCompressedStepForRawStep only at block granularity - both startRawPos and rawSeekBytes should be multiples of OODLELZ_BLOCK_LEN (except at the end of the stream). As long as you do that, then pEndRawPos will = startRawPos + rawSeekBytes.

You can use it at quantum granularity (OODLELZ_QUANTUM_LEN), but there are some caveats. You cannot step quanta inside uncompressed blocks, only in normal LZ blocks. If you try to seek quanta inside an uncompressed block, you will get pEndRawPos = the end of the block.

You can only resume seeking from pEndRawPos .

returns 0 for valid not-enough-data case returns -1 for error

If compAvail is not the whole compressed buffer, then the returned step may be less than the amount you requested. eg. if the compressed data in compAvail does not contain enough data to make a step of rawSeekBytes a smaller step will be taken. NOTE : can return comp step > comp avail !


 

OodleLZ_GetInPlaceDecodeBufferSizeOodleAPI_LZ_CompressorsOodleLZ_GetAllChunksCompressor

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Welcome to Oodle
Navigation
 Welcome to Oodle
 Change Log

Welcome to Oodle2

Oodle is a family of solutions for efficient data compression. Oodle comes as four separate SDKs :

  • Oodle Data : generic lossless data compression (Kraken, Leviathan, Mermaid, and Selkie)
  • Oodle Network : packet compression to reduce bandwidth.
  • Oodle Texture : encoding for BCN block-compressed GPU textures that dramatically reduces their size.
  • Oodle Lossless Image : specialized encoder for lossless RGB image encoding. Faster and smaller than PNG.

Oodle Data and Network are described here. Oodle Texture and Oodle Lossless Image are available from RAD.

Getting Started with Oodle LZ Data Compression

Getting Started with Oodle Network

See About Oodle for more. Also check out the Examples.

Table of Contents

  •  Welcome to Oodle
  •  About Oodle
    •  Index of Abouts
    •  About Oodle ozip
    •  About Oodle on Platforms
      •  About Oodle on Windows
        •  About Oodle on Windows UWP
      •  About Oodle on PS4
      •  About Oodle on Nintendo Switch
      •  About Oodle on Mac
      •  About Oodle on Xbox One
      •  About Oodle on Linux
      •  About Oodle on IOS
      •  About Oodle on Android
      •  About Oodle on WASM
    •  About Oodle Job Threading Plugins
    •  About Compression Scratch Memory
  •  Frequently Asked Questions
    •  FAQ: OodleLZ_Decompress is failing how do I diagnose it?
    •  FAQ: How do I use Oodle with no allocator?
    •  FAQ: How do I limit the encoder memory use?
    •  FAQ: How much memory do the Oodle compressors use ?
    •  FAQ: What are the Oodle deprecated compressors ?
    •  FAQ: Do new Oodle versions break data compatibility ?
    •  FAQ: How does OodleLZ compare with other compressors ?
    •  FAQ: Which OodleLZ should I use?
    •  FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
    •  FAQ: How do I decompress to graphics memory quickly?
    •  FAQ: How do I get the Oodle logs?
    •  FAQ: I write a file with IOQ but the contents are garbage?
    •  FAQ: I ran out of OodleHandle table slots; what do I do?
    •  FAQ: What is SINTa? How do I load files bigger than 2 GB?
    •  FAQ: My Files aren't loading right and I can't track it down
  •  Oodle2 Core API Documentation
    •  Core Base
      •  Defines
        •  OODLE_MALLOC_MINIMUM_ALIGNMENT
        •  OODLE_JOB_MAX_DEPENDENCIES
        •  OODLE_JOB_NULL_HANDLE
        •  t_fp_Oodle_Job
        •  OODLE_HEADER_VERSION
        •  OodleNetworkVersion
      •  Enumerants
        •  Oodle_UsageWarnings
      •  Structures
        •  OodleConfigValues
      •  Functions
        •  Oodle_GetConfigValues
        •  Oodle_SetConfigValues
        •  Oodle_SetUsageWarnings
        •  Oodle_CheckVersion
        •  Oodle_LogHeader
      •  Typedefs
        •  t_OodleFPVoidVoid
        •  t_OodleFPVoidVoidStar
    •  Core plugins
      •  Functions
        •  OodleCore_Plugins_SetAllocators
        •  OodleCore_Plugins_SetJobSystem
        •  OodleCore_Plugins_SetJobSystemAndCount
        •  OodleCore_Plugins_SetPrintf
        •  OodleCore_Plugins_SetAssertion
      •  Typedefs
        •  t_fp_OodleCore_Plugin_MallocAligned
        •  t_fp_OodleCore_Plugin_Free
        •  t_fp_OodleCore_Plugin_RunJob
        •  t_fp_OodleCore_Plugin_WaitJob
        •  t_fp_OodleCore_Plugin_Printf
        •  t_fp_OodleCore_Plugin_DisplayAssertion
    •  Core LZ compression
      •  About OodleLZ
      •  About OodleLZ ThreadPhased Decode
      •  About OodleLZ Hydra
      •  OodleAPI_LZ_Compressors
        •  Defines
          •  OODLE_ALLOW_DEPRECATED_COMPRESSORS
          •  OODLELZ_LOCALDICTIONARYSIZE_MAX
          •  OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
          •  OODLELZ_BLOCK_LEN
          •  OODLELZ_BLOCK_MAX_COMPLEN
          •  OODLELZ_QUANTUM_LEN
          •  OODLELZ_FAILED
          •  OODLELZ_SCRATCH_MEM_NO_BOUND
        •  Enumerants
          •  OodleLZ_Verbosity
          •  OodleLZ_Compressor
          •  OodleLZ_PackedRawOverlap
          •  OodleLZ_CheckCRC
          •  OodleLZ_Profile
          •  OodleDecompressCallbackRet
          •  OodleLZ_CompressionLevel
          •  OodleLZ_Jobify
          •  OodleLZ_Decode_ThreadPhase
          •  OodleLZ_FuzzSafe
          •  OodleLZSeekTable_Flags
          •  OodleLZ_CompressScratchMemBoundType
        •  Structures
          •  OodleLZ_CompressOptions
          •  OodleLZ_DecodeSome_Out
          •  OodleLZDecoder
          •  OodleLZ_SeekTable
        •  Functions
          •  OodleLZ_Compress
          •  OodleLZ_Decompress
          •  OodleLZDecoder_Create
          •  OodleLZDecoder_MemorySizeNeeded
          •  OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
          •  OodleLZDecoder_Destroy
          •  OodleLZDecoder_Reset
          •  OodleLZDecoder_DecodeSome
          •  OodleLZDecoder_MakeValidCircularWindowSize
          •  OodleLZ_MakeSeekChunkLen
          •  OodleLZ_GetNumSeekChunks
          •  OodleLZ_GetSeekTableMemorySizeNeeded
          •  OodleLZ_FillSeekTable
          •  OodleLZ_CreateSeekTable
          •  OodleLZ_FreeSeekTable
          •  OodleLZ_CheckSeekTableCRCs
          •  OodleLZ_FindSeekEntry
          •  OodleLZ_GetSeekEntryPackedPos
          •  OodleLZ_CompressionLevel_GetName
          •  OodleLZ_Compressor_GetName
          •  OodleLZ_Jobify_GetName
          •  OodleLZ_CompressOptions_GetDefault
          •  OodleLZ_CompressOptions_Validate
          •  OodleLZ_Compressor_UsesWholeBlockQuantum
          •  OodleLZ_Compressor_UsesLargeWindow
          •  OodleLZ_Compressor_CanDecodeInCircularWindow
          •  OodleLZ_Compressor_CanDecodeThreadPhased
          •  OodleLZ_Compressor_CanDecodeInPlace
          •  OodleLZ_Compressor_MustDecodeWithoutResets
          •  OodleLZ_Compressor_CanDecodeFuzzSafe
          •  OodleLZ_Compressor_RespectsDictionarySize
          •  OodleLZ_GetCompressScratchMemBound
          •  OodleLZ_GetCompressScratchMemBoundEx
          •  OodleLZ_GetCompressedBufferSizeNeeded
          •  OodleLZ_GetDecodeBufferSize
          •  OodleLZ_GetInPlaceDecodeBufferSize
          •  OodleLZ_GetCompressedStepForRawStep
          •  OodleLZ_GetAllChunksCompressor
          •  OodleLZ_GetFirstChunkCompressor
          •  OodleLZ_GetChunkCompressor
        •  Typedefs
          •  OodleDecompressCallback
  •  Oodle2 Ext API Documentation
    •  OodleX LZ compression
      •  OodleXAPI_LZ_Async
        •  Structures
          •  OodleDecompressCallback_WriteFile_Data
        •  Functions
          •  OodleXLZ_Decompress_ThreadPhased_Narrow_Async
          •  OodleXLZ_Decompress_Narrow_Async
          •  OodleXLZ_ReadAndDecompress_Wide_Async
          •  OodleXLZ_Decompress_Wide_Async
          •  OodleXLZ_Decompress_MakeSeekTable_Wide_Async
          •  OodleXLZ_Compress_Async
          •  OodleXLZ_Compress_Wait_GetResult
          •  OodleXLZ_Compress_AsyncAndWait
          •  OodleXLZ_ReadAndDecompress_Stream_Async
          •  OodleXDecompressCallback_WriteFile_Data_Init
          •  OodleDecompressCallback_WriteFile
    •  OodleX Startup and Shutdown
      •  Defines
        •  OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
        •  OODLE_WORKERS_COUNT_ALL_HYPER_CORES
      •  Enumerants
        •  OodleX_Init_GetDefaults_DebugSystems
        •  OodleX_Init_GetDefaults_Threads
        •  OodleX_Shutdown_LogLeaks
        •  OodleX_Shutdown_DebugBreakOnLeaks
      •  Structures
        •  OodleXInitOptions
        •  OodleXConfigValues
      •  Functions
        •  OodleX_Init_ThreadProfilerInit
        •  OodleX_Init_GetDefaults
        •  OodleX_Init_GetDefaults_Minimal
        •  OodleX_Init
        •  OodleX_Init_Default
        •  OodleX_LogSystemInfo
        •  OodleX_Shutdown
        •  OodleX_Init_NoThreads
        •  OodleX_Shutdown_NoThreads
        •  OodleX_GetConfigValues
        •  OodleX_SetConfigValues
    •  OodleX Memory Allocators
      •  About OodleXMalloc
      •  Enumerants
        •  OodleXMalloc_OS_Options
      •  Structures
        •  OodleXMallocVTable
      •  Functions
        •  OodleXMalloc_InstallVTable
        •  OodleXMalloc_SetFailedHandler
        •  OodleXMalloc
        •  OodleXMallocAligned
        •  OodleXFree
        •  OodleXFreeSized
        •  OodleXMallocBigAlignment
        •  OodleXMallocBig
        •  OodleXFreeBig
        •  OodleXMalloc_ValidatePointer
        •  OodleXMalloc_IOAligned
        •  OodleXFree_IOAligned
        •  OodleXMalloc_GetVTable_Clib
        •  OodleXMalloc_GetVTable_OS
      •  Typedefs
        •  OodleXMallocFailedHandler
    •  OodleX async handle operations
      •  About OodleXHandle
      •  Defines
        •  OODLEX_ASYNC_HANDLE_INVALID
        •  OODLEX_ASYNC_HANDLE_PENDING
        •  OODLEX_ASYNC_HANDLE_DONE
        •  OODLEX_ASYNC_HANDLE_ERROR
      •  Enumerants
        •  OodleXPriority
        •  OodleXAsyncSelect
        •  OodleXStatus
        •  OodleXHandleAutoDelete
        •  OodleXHandleKickDelayed
        •  OodleXHandleDeleteIfDone
      •  Functions
        •  OodleX_GetStatus
        •  OodleX_Wait
        •  OodleX_WaitAll
        •  OodleX_WaitDoneAllPending
        •  OodleX_SetHandleAutoDelete
        •  OodleX_GetAvailableAsyncSelect
        •  OodleXHandleEvent_Alloc
        •  OodleXHandleEvent_SetDone
        •  OodleXHandleEvent_SetError
        •  OodleXHandleCountdown_Alloc
        •  OodleXHandleCountdown_Decrement
      •  Typedefs
        •  OodleXHandle
    •  OodleX threading util
      •  Functions
        •  OodleX_Semaphore_Post
        •  OodleX_Semaphore_Wait
        •  OodleX_CreateThread
        •  OodleX_WaitAndDestroyThread
        •  OodleX_ReleaseThreadTLS
        •  OodleX_CorePlugin_RunJob
        •  OodleX_CorePlugin_WaitJob
        •  OodleX_GetNumWorkerThreads
      •  Typedefs
        •  OodleX_Semaphore
        •  OodleX_ThreadFunc
    •  OodleX low level async io
      •  About OodleIOQ
      •  Defines
        •  OODLEX_IO_MAX_ALIGNMENT
        •  OODLEX_BUFFER_SIZE_DEFAULT
        •  OODLEX_FILEINFO_FLAG_INVALID
        •  OODLEX_FILEINFO_MODTIME_INVALID
        •  OODLEX_FILE_SIZE_INVALID
        •  OODLEX_FILE_OPEN_NO_RESERVE_SIZE
        •  OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
      •  Enumerants
        •  OodleXCopyFileFlags
        •  OodleFileNotFoundIsAnError
        •  OODLEX_FILEINFO_FLAGS
        •  OodleXError
        •  OodleXFileMode
        •  OodleXFileOpenFlags
      •  Structures
        •  OodleXFileOpsVTable
        •  OodleXFileInfo
      •  Functions
        •  OodleXIOQ_WaitDoneAllPending
        •  OodleXIOQ_KickAnyDelayed
        •  OodleXIOQ_GetStatus
        •  OodleXIOQ_GetErrorDetails
        •  OodleXIOQ_GetErrorEnum
        •  OodleXIOQ_LogError
        •  OodleXIOQ_Wait
        •  OodleXIOQ_GetInfo
        •  OodleXIOQ_Wait_GetInfo
        •  OodleXIOQ_GetLastPendingOpOnFile
        •  OodleXIOQ_GetName
        •  OodleXIOQ_GetLastError
        •  OodleXIOQ_ClearError
        •  OodleXIOQ_LogLastError
        •  OodleXIOQ_GetOSHandle
        •  OodleXIOQ_SetVTable
        •  OodleXIOQ_Fence_Async
        •  OodleXIOQ_OpenForRead_Async
        •  OodleXIOQ_OpenAndRead_Async
        •  OodleXIOQ_OpenForWriteCreate_Async
        •  OodleXIOQ_OpenForWriteTempName_Async
        •  OodleXIOQ_CloseFile_Async
        •  OodleXIOQ_CloseFileRename_Async
        •  OodleXIOQ_Read_Async
        •  OodleXIOQ_Write_Async
        •  OodleXIOQ_SetFileSize_Async
        •  OodleXIOQ_ReserveFileSizeForWrite_Async
        •  OodleXIOQ_ForceWriteable_Async
        •  OodleXIOQ_Delete_Async
        •  OodleXIOQ_Rename_Async
        •  OodleXIOQ_MakeDir_Async
        •  OodleXIOQ_FreeBufferIOAligned_Async
        •  OodleXIOQ_GetInfoByName_Async
        •  OodleXIOQ_GetInfoByName_GetResult
        •  OodleXIOQ_SetInfoByName_Async
        •  OodleXIOQ_ReadMallocWholeFile_Async
        •  OodleXIOQ_ReadMallocWholeFile_GetResult
        •  OodleXIOQ_OpenAndReadMallocWholeFile_Async
        •  OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
        •  OodleXIOQ_OpenWriteWholeFileClose_Async
        •  OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
        •  OodleXIOQ_ReadUnalignedAdjustPointer_Async
        •  OodleXIOQ_MakeAllDirs_Async
        •  OodleXIOQ_CopyFile_Async
        •  OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
        •  OodleXIOQ_WriteWholeFile_AsyncAndWait
        •  OodleXIOQ_CopyFile_AsyncAndWait
        •  OodleXIOQ_GetInfoByName_AsyncAndWait
        •  OodleXIOQ_SetInfoByName_AsyncAndWait
        •  OodleXIOQ_MakeAllDirs_AsyncAndWait
        •  OodleXIOQ_Delete_AsyncAndWait
        •  OodleXIOQ_Rename_AsyncAndWait
        •  OodleXIOQ_GetFileSize_AsyncAndWait
        •  OodleXIOQ_NameIsDir_AsyncAndWait
        •  OodleX_GetOSFileOps
        •  OodleX_GetDefaultFileOps
        •  OodleX_SetDefaultFileOps
      •  Typedefs
        •  OodleXIOQFile
        •  OodleXOSFile
        •  OodleXOSFileListing
    •  OodleX Debug aids
      •  Defines
        •  OodleXLog_Printf
      •  Enumerants
        •  OodleXLog_StateFlags
        •  OodleXLog_VerboseLevel
        •  OodleXLogCallbackRetRet
      •  Functions
        •  OodleXLog_SetState
        •  OodleXLog_SetEcho
        •  OodleXLog_GetEcho
        •  OodleXLog_SetCallback
        •  OodleXLog_GetCallback
        •  OodleXLog_GetVerboseLevel
        •  OodleXLog_SetVerboseLevel
        •  OodleXLog_Flush
        •  OodleXLog_PrintfError
      •  Typedefs
        •  OodleXLogCallbackRet
    •  OodleX Utils
      •  Defines
        •  OODLEX_PATH_DELIM
      •  Functions
        •  OodleX_GetExtensionKey
        •  OodleX_MakeExtensionKey
        •  OodleX_IOAlignUpS32
        •  OodleX_IOAlignUpS64
        •  OodleX_IOAlignUpSINTa
        •  OodleX_IOAlignDownS32
        •  OodleX_IOAlignDownS64
        •  OodleX_IOAlignDownSINTa
        •  OodleX_S64_to_SINTa_check
        •  OodleX_GetSeconds
        •  OodleXUtil_ConvertUTF8ToUTF16
        •  OodleXUtil_ConvertUTF16ToUTF8
        •  OodleX_CombinePaths
        •  OodleX_GetOSCwd
        •  OodleX_PrefixOSCwd
    •  About Oodle Ext
  •  Oodle2 Network API Documentation
    •  Oodle Network compression
      •  OodleAPI_OodleNetwork1
        •  About OodleNetwork1
        •  Defines
          •  OODLENETWORK1_MAX_DICTIONARY_SIZE
          •  OODLENETWORK1_HASH_BITS_DEFAULT
          •  OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
        •  Structures
          •  OodleNetwork1_Shared
          •  OodleNetwork1TCP_State
          •  OodleNetwork1UDP_State
          •  OodleNetwork1UDP_StateCompacted
        •  Functions
          •  OodleNetwork1_Shared_Size
          •  OodleNetwork1TCP_State_Size
          •  OodleNetwork1_CompressedBufferSizeNeeded
          •  OodleNetwork1_Shared_SetWindow
          •  OodleNetwork1TCP_State_Reset
          •  OodleNetwork1TCP_State_InitAsCopy
          •  OodleNetwork1TCP_Train
          •  OodleNetwork1TCP_Encode
          •  OodleNetwork1TCP_Decode
          •  OodleNetwork1UDP_Train
          •  OodleNetwork1UDP_State_Size
          •  OodleNetwork1UDP_Encode
          •  OodleNetwork1UDP_Decode
          •  OodleNetwork1UDP_StateCompacted_MaxSize
          •  OodleNetwork1UDP_State_Compact_ForVersion
          •  OodleNetwork1UDP_State_Compact
          •  OodleNetwork1UDP_State_Uncompact_ForVersion
          •  OodleNetwork1UDP_State_Uncompact
          •  OodleNetwork1_SelectDictionarySupported
          •  OodleNetwork1_SelectDictionaryFromPackets
          •  OodleNetwork1_SelectDictionaryFromPackets_Trials
      •  About Oodle Network Compression
        •  Capturing Training data for OodleNetwork
        •  Forming Packets for Maximum Compression
    •  Network plugins
      •  Functions
        •  OodleNet_Plugins_SetAllocators
        •  OodleNet_Plugins_SetJobSystem
        •  OodleNet_Plugins_SetJobSystemAndCount
        •  OodleNet_Plugins_SetPrintf
        •  OodleNet_Plugins_SetAssertion
      •  Typedefs
        •  t_fp_OodleNet_Plugin_MallocAligned
        •  t_fp_OodleNet_Plugin_Free
        •  t_fp_OodleNet_Plugin_RunJob
        •  t_fp_OodleNet_Plugin_WaitJob
        •  t_fp_OodleNet_Plugin_Printf
        •  t_fp_OodleNet_Plugin_DisplayAssertion
  •  Examples
    •  How to build and use the Oodle examples
    •  example_lz : Example demonstrating LZ compression and decompression
    •  example_lz_chart : Example that makes a chart of OodleLZ options
    •  example_lz_noallocs : Example demonstrating Oodle compression with no allocations
    •  example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
    •  example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
    •  example_lz_threadphased : Example of 2-thread ThreadPhased decoding
    •  example_network_client : Example with simple network client support
    •  example_packet : Example demonstrating network packet compression
  •  Acknowledgements
    •  Third Party License Notices
  •  Change Log
  •  Getting Started with Oodle Network
  •  Introducing the new Oodle Leviathan
  •  Oodle2 Core vs Oodle2 Ext
  •  Tips for benchmarking a compressor
  •  Getting Started with Oodle LZ Data Compression
  •  Oodle Network and Data SDK separation

 
   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressScratchMemBoundType
Navigation
 OodleLZSeekTable_Flags
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_CompressScratchMemBoundType
{
    OodleLZ_CompressScratchMemBoundType_WorstCase = 0,
    OodleLZ_CompressScratchMemBoundType_Typical = 1,
    OodleLZ_CompressScratchMemBoundType_Force32 = 0x40000000
};
Discussion
Type of scratch memory bound to compute.
Enumerants
OodleLZ_CompressScratchMemBoundType_WorstCase  Worst-case memory use estimate
OodleLZ_CompressScratchMemBoundType_Typical  Typical memory use estimate; with this much scratch memory, allocations might still happen, but rarely.
OodleLZ_CompressScratchMemBoundType_Force32 
Discussion
OodleLZ_CompressScratchMemBoundType_WorstCase provides a worst-case estimate for the amount of scratch memory required for compression. If at least this much scratch memory is provided to the compression functions, they are guaranteed to make no allocations of their own. Some compression levels don't have simple worst-case memory use guarantees and will return OODLELZ_SCRATCH_MEM_NO_BOUND for this type of query.

OodleLZ_CompressScratchMemBoundType_Typical returns a scratch memory estimate that will be sufficient for most compression calls to not require additional memory, even when no hard guarantees exist. Even so, a good estimate may not be available, in which case OODLELZ_SCRATCH_MEM_NO_BOUND is returned.
 

OodleLZSeekTable_FlagsOodleAPI_LZ_CompressorsOodleLZ_CompressOptions

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenAndRead_Async
Navigation
 OodleXIOQ_OpenForRead_Async
 OodleXIOQ_OpenForWriteCreate_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenAndRead_Async( OodleXIOQFile * pFile,
                                          const char * name,
                                          void * initialReadMemory,
                                          OO_SINTa initialReadSize,
                                          OO_S64 initialReadPos OODEFAULT( 0 ),
                                          OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                          const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                          OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                          OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                          const OodleXHandle * dependencies OODEFAULT( NULL ),
                                          OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start opening a file for read, and do an initial read
Parameters
pFile  filled with a handle to the file which will be opened
name  name of the file to open (VFS, UTF-8)
initialReadMemory  pointer to buffer to read into (must be OODLEX_IO_MAX_ALIGNMENT aligned)
initialReadSize  amount to read (must be OODLEX_IO_MAX_ALIGNMENT aligned)
initialReadPos  (optional) file position to read (must be OODLEX_IO_MAX_ALIGNMENT aligned)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

If vtable is NULL, the vtable specified by the VFS mapping is used. The file name provided is automatically run through VFS-to-OS name mapping, if applicable.

Open returns a File ref right away for your convenience, but the file is not actually open for a little while. You can however go ahead and queue more requests on the file reference before open is complete. You cannot call things that require an open file, such as OodleXIOQ_GetInfo. OpenForRead is always shared access (when possible).

Also performs an initial read. Particularly useful when you need an initial header before you can start processing a file.


 

OodleXIOQ_OpenForRead_AsyncOodleX low level async ioOodleXIOQ_OpenForWriteCreate_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Introducing the new Oodle Leviathan
Navigation
 Welcome to Oodle
 Change Log

Oodle Leviathan is a new compressor that provides higher compression than Oodle Kraken while still achieving very high decode speed.

Kraken revolutionized what was possible in lossless compression, providing insane decode speeds (3-4X faster than zlib) with still excellent ratios. Leviathan does the same for the higher compression domain, still 2-3X faster than zlib but with ratios that compete with or beat LZMA/7zip.

If you loved Kraken but want a little bit more compression ratio, Leviathan is for you. If you like the high ratio of LZMA/7zip or Oodle's LZNA, but want more decode speed, Leviathan is for you.

Leviathan is way faster to decode than anything else in the world that gets as much compression. It's over 10X faster to decode than LZMA/7zip.

For example, on the "seven" testset :

ooLeviathan7:  3.23:1 ,    0.9 enc MB/s ,  642.4 dec MB/s
lzmadef9    :  3.18:1 ,    3.0 enc MB/s ,   53.8 dec MB/s
ooKraken7   :  3.09:1 ,    1.3 enc MB/s ,  948.5 dec MB/s
ooMermaid7  :  2.85:1 ,    2.1 enc MB/s , 1704.1 dec MB/s
zlib9       :  2.33:1 ,    7.7 enc MB/s ,  309.0 dec MB/s

Leviathan is currently rather slow to encode (it does extensive optimization of the bit stream), so if you're interested in high speed encoding, Leviathan is not the answer. Kraken & Mermaid both provide fast encoding.

See for yourself how great Leviathan is by trying example_lz_chart : Example that makes a chart of OodleLZ options

Dive in to Getting Started with Oodle LZ Data Compression
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_GetConfigValues
Navigation
 Oodle_SetConfigValues
 Welcome to Oodle
 Change Log
// Function prototype:
void Oodle_GetConfigValues( OodleConfigValues * ptr );
Discussion
Get OodleConfigValues
Parameters
ptr  filled with OodleConfigValues
Discussion

Gets the current OodleConfigValues.

May be different per platform.
 

OodleConfigValuesCore BaseOodle_SetConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: I ran out of OodleHandle table slots; what do I do?
Navigation
 FAQ: I write a file with IOQ but the contents are garbage?
 Frequently Asked Questions
 FAQ: What is SINTa? How do I load files bigger than 2 GB?
 Welcome to Oodle
 Change Log

This is for OodleX async helpers, does not apply to Oodle Core.

Since Oodle 2.9.7, running out of handles is nearly impossible and the default maximum number of handles is much higher. The handle table is now allocated in chunks of 2048, with the config m_num_handles_log2 setting the number of chunks. So the default m_num_handles_log2 of 13 means 2^13 chunks of 2048, or 2^24 maximum handles total.

By default, Oodle provides 2^24 slots for async handles. This is the maximum number of handles which can be alive at any time, but there is no limit on the number of completed or deleted handles.

If you run out of handles, it's usually due to a bug in usage that you should fix, you generally don't need to change the size of the handle table. Some possibilities :

1. You might be leaking handles. That is, you fired off some async operation with an OodleXHandleAutoDelete of OodleXHandleAutoDelete_No , but then you didn't delete the handle (eg. by calling Oodle_Wait() with OodleXHandleDeleteIfDone_Yes. You should either change the lifetime to OodleXHandleAutoDelete_Yes if you don't care about checking on the completion status of the handle, or you should make sure to delete the handle at some point.

2. You might really need all those async handles. Simply change OodleXInitOptions:m_num_handles_log2 before calling OodleX_Init.
 

FAQ: I write a file with IOQ but the contents are garbage?Frequently Asked QuestionsFAQ: What is SINTa? How do I load files bigger than 2 GB?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetInfoByName_AsyncAndWait
Navigation
 OodleXIOQ_CopyFile_AsyncAndWait
 OodleXIOQ_SetInfoByName_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_GetInfoByName_AsyncAndWait( const char * vfsName,
                                              OodleXFileInfo * pInfo,
                                              OodleFileNotFoundIsAnError fnfiae );
Discussion
See OodleXIOQ_GetInfoByName_Async
Discussion
OodleXIOQ_GetInfoByName_AsyncAndWait returns true if the file was found and info was retrieved successfully. The return value is always false for file not found, even if you pass OodleFileNotFoundIsAnError_No.
 
OodleXIOQ_CopyFile_AsyncAndWaitOodleX low level async ioOodleXIOQ_SetInfoByName_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Compress_Async
Navigation
 OodleXLZ_Decompress_MakeSeekTable_Wide_Async
 OodleXLZ_Compress_Wait_GetResult
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_Compress_Async( OO_U32 asyncSelect,
                                      OodleLZ_Compressor compressor,
                                      const void * rawBuf,
                                      OO_SINTa rawLen,
                                      void * compBuf,
                                      OodleLZ_CompressionLevel compressSelect,
                                      const OodleLZ_CompressOptions * pOptions OODEFAULT( NULL ),
                                      const void * dictionaryBase OODEFAULT( NULL ),
                                      OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                      const OodleXHandle * dependencies OODEFAULT( NULL ),
                                      OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an async LZ compress
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run
compressor  A member of OodleLZ_Compressor to select the compressor
rawBuf  raw data to compress
rawLen  amount of raw data to compress
compBuf  output compressed data
compressSelect  A member of OodleLZ_CompressionLevel to select the compression level
pOptions  (optional) compression options
dictionaryBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  OodleXHandle to the operation, or OodleXHandle_Null for invalid arguments
Discussion

Runs "wide" if asyncFlags includes OodleXAsyncSelect_Workers + OodleXAsyncSelect_Wide.

The output compressed data can be decompressed "wide" if pOptions set seekChunkReset = true.

"wide" means use many threads at once on this single operation. If the compression is selected to run wide, but the decompression cannot run wide (eg. the compressed data does not have small independent chunks), then it will still compress wide, but on a very large granularity, instead of the small OODLELZ_BLOCK_LEN granularity). In that case, only very large buffers will be compressed in parallel.

Use OodleXLZ_Compress_Wait_GetResult to get the result and free associated structures. OodleXLZ_Compress_Wait_GetResult must be called even if you don't want the result, or memory will be leaked.
 

OodleXLZ_Decompress_MakeSeekTable_Wide_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Compress_Wait_GetResult

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz_chart : Example that makes a chart of OodleLZ options
Navigation
 example_lz : Example demonstrating LZ compression and decompression
 Examples
 example_lz_noallocs : Example demonstrating Oodle compression with no allocations
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz_chart

The Oodle SDK comes with a pre-built exe for example_lz_chart in the bin/ directory

usage : example_lz_chart

Run with a file name, which will be loaded and used as data to test on.

You can also toggle compile-time options with the define EXAMPLE_LZ_CHART_NUM_LEVELS below.

makes an output like this :

Oodle 2.6.3 example_lz_chart 
lz_chart loading r:testsetslztestsetlzt99...
file size : 24700820
------------------------------------------------------------------------------
Selkie : super fast to encode & decode, least compression
Mermaid: fast decode with better-than-zlib compression
Kraken : good compression, fast decoding, great tradeoff!
Leviathan : very high compression, slowest decode
------------------------------------------------------------------------------
chart cell shows | raw/comp ratio : encode MB/s : decode MB/s |
All compressors run at various encoder effort levels (SuperFast - Optimal).
Many repetitions are run for accurate timing.
------------------------------------------------------------------------------
       |   HyperFast4|   HyperFast3|   HyperFast2|   HyperFast1|   SuperFast |
Selkie |1.41:675:3895|1.45:622:3888|1.53:465:3696|1.68:369:3785|1.70:342:3759|
Mermaid|1.66:436:2189|1.66:436:2188|1.79:352:2090|2.01:276:2055|2.04:261:2025|
Kraken |1.55:588:1839|1.71:419:1136|1.88:331:1087|2.10:279:1093|2.27:167:1010|
------------------------------------------------------------------------------
compression ratio (raw/comp):
       |   HyperFast4|   HyperFast3|   HyperFast2|   HyperFast1|   SuperFast |
Selkie |    1.412    |    1.447    |    1.526    |    1.678    |    1.698    |
Mermaid|    1.660    |    1.660    |    1.793    |    2.011    |    2.041    |
Kraken |    1.548    |    1.711    |    1.877    |    2.103    |    2.268    |
------------------------------------------------------------------------------
encode speed (MB/s):
       |   HyperFast4|   HyperFast3|   HyperFast2|   HyperFast1|   SuperFast |
Selkie |    674.548  |    621.811  |    464.555  |    369.364  |    341.588  |
Mermaid|    435.650  |    435.923  |    352.475  |    276.199  |    260.511  |
Kraken |    588.488  |    418.921  |    331.423  |    279.129  |    167.206  |
------------------------------------------------------------------------------
decode speed (MB/s):
       |   HyperFast4|   HyperFast3|   HyperFast2|   HyperFast1|   SuperFast |
Selkie |   3894.644  |   3887.820  |   3695.984  |   3785.457  |   3758.594  |
Mermaid|   2189.030  |   2187.863  |   2090.319  |   2054.897  |   2024.692  |
Kraken |   1839.091  |   1135.920  |   1086.922  |   1093.407  |   1009.967  |
------------------------------------------------------------------------------
       |   VeryFast  |   Fast      |   Normal    |   Optimal1  |   Optimal3  |
Selkie |1.75:205:3490|1.83:105:3687|1.86: 43:3815|1.93:5.1:3858|1.94:2.6:3856|
Mermaid|2.12:173:1991|2.19: 84:2177|2.21: 32:2291|2.37:2.8:2058|2.44:1.8:1978|
Kraken |2.32:112:1104|2.39: 37:1187|2.43: 20:1189|2.55:3.1:1103|2.65:1.2:1038|
Leviath|2.50: 31: 738|2.57: 17: 787|2.62:9.5: 807|2.71:1.6: 811|2.76:0.9: 776|
------------------------------------------------------------------------------
compression ratio (raw/comp):
       |   VeryFast  |   Fast      |   Normal    |   Optimal1  |   Optimal3  |
Selkie |    1.748    |    1.833    |    1.863    |    1.933    |    1.943    |
Mermaid|    2.118    |    2.194    |    2.207    |    2.367    |    2.437    |
Kraken |    2.320    |    2.390    |    2.434    |    2.551    |    2.646    |
Leviath|    2.504    |    2.572    |    2.617    |    2.707    |    2.756    |
------------------------------------------------------------------------------
encode speed (MB/s):
       |   VeryFast  |   Fast      |   Normal    |   Optimal1  |   Optimal3  |
Selkie |    204.621  |    104.758  |     42.504  |      5.102  |      2.554  |
Mermaid|    172.681  |     84.227  |     32.030  |      2.798  |      1.836  |
Kraken |    111.858  |     37.126  |     19.859  |      3.091  |      1.204  |
Leviath|     31.031  |     16.697  |      9.461  |      1.621  |      0.869  |
------------------------------------------------------------------------------
decode speed (MB/s):
       |   VeryFast  |   Fast      |   Normal    |   Optimal1  |   Optimal3  |
Selkie |   3490.442  |   3686.689  |   3814.655  |   3857.857  |   3856.226  |
Mermaid|   1991.442  |   2176.725  |   2291.498  |   2057.575  |   1977.721  |
Kraken |   1104.172  |   1186.638  |   1189.372  |   1103.148  |   1038.352  |
Leviath|    737.934  |    787.152  |    806.523  |    811.161  |    775.800  |
------------------------------------------------------------------------------

#include "../include/oodle2x.h"
#include "ooex.h"

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz_chart
#endif

//===========================================================
// NOTE : these timings are hot cache (no cache invalidation between repeats)
//  that could be significant on very small buffers

struct time_and_len
{
    double time;
    OO_SINTa len;
};

// if scratch is NULL or insufficient, Oodle will allocate internally
static void * s_scratch_memory = NULL;
static OO_SINTa s_scratch_memory_size = 0;

static time_and_len Encode_And_Time(const void * raw_buf, OO_SINTa raw_len, void * comp_buf,
    OodleLZ_Compressor compressor,
    OodleLZ_CompressionLevel compression_level,
    OodleLZ_CompressOptions * compression_options,
    int min_repeats,
    double min_total_seconds)
{
    double total_seconds = 0;
    time_and_len ret;
    ret.time = 999999.9;

    for(;;)
    {
        double dt = OodleX_GetSeconds();
        
        ret.len = OodleLZ_Compress(compressor, raw_buf, raw_len, comp_buf, compression_level, compression_options,
            NULL, NULL, s_scratch_memory, s_scratch_memory_size);

        dt = OodleX_GetSeconds() - dt;
        
        total_seconds += dt;
        ret.time = OOEX_MIN(ret.time,dt);
        min_repeats--;
        
        if ( min_repeats <= 0 && total_seconds >= min_total_seconds )
            break;
    }
    
    return ret;
}

static double Decode_And_Time( void * comp_buf, OO_SINTa comp_len, void * decode_buffer, OO_SINTa in_size,
    int min_repeats,
    double min_total_seconds)
{
    double total_seconds = 0;
    double ret = 999999.9;

    for(;;)
    {
        double dt = OodleX_GetSeconds();
                
        OO_SINTa decode_len = OodleLZ_Decompress( comp_buf, comp_len, decode_buffer, in_size , OodleLZ_FuzzSafe_Yes,
            OodleLZ_CheckCRC_No, OodleLZ_Verbosity_None, NULL, 0, NULL,NULL, s_scratch_memory,s_scratch_memory_size );

        dt = OodleX_GetSeconds() - dt;
        
        OOEX_ASSERT_ALWAYS( decode_len == in_size );
        
        total_seconds += dt;
        ret = OOEX_MIN(ret,dt);
        min_repeats--;
        
        if ( min_repeats <= 0 && total_seconds >= min_total_seconds )
            break;
    }
    
    return ret;
}
        
//===========================================================

static void bar()
{
    OodleXLog_Printf_v1("------------------------------------------------------------------------------\n");
}

static void header(const OodleLZ_CompressionLevel * chart_levels, int num_levels)
{
    //OodleXLog_Printf_v1("%5s |   %-10s|   %-10s|   %-10s|   %-10s|\n",
    //  "","VeryFast","Fast","Normal","Optimal1");
        
    OodleXLog_Printf_v1("%7s|","");
    
    for(int l=0;l<num_levels;l++)
    {           
        OodleLZ_CompressionLevel level = chart_levels[l];
        
        OodleXLog_Printf_v1("   %-10s|",
           OodleLZ_CompressionLevel_GetName(level));
    }
    OodleXLog_Printf_v1("\n");
}

static const char * truncated_compressor_name(  OodleLZ_Compressor compressor )
{
    const char * n = OodleLZ_Compressor_GetName(compressor);
    static char buf[10];
    memset(buf,' ',sizeof(buf));
    memcpy(buf,n,OOEX_MIN( strlen(n), sizeof(buf) ) );
    buf[7] = 0;
    return buf;
}   
        
extern "C" int main(int argc,char *argv[])
{
    // Init Oodle systems with default options :
    OodleXInitOptions opts;
    if ( ! OodleX_Init_GetDefaults(OODLE_HEADER_VERSION,&opts) )
    {
        fprintf(stderr,"Oodle header version mismatch.\n");
        return 10;
    }
    // change opts here if you like
    // NOTE : default options enable the OodleX thread system
    //  so encoders will be "Jobified"
    if ( ! OodleX_Init(OODLE_HEADER_VERSION,&opts) )
    {
        fprintf(stderr,"OodleX_Init failed.\n");
        return 10;
    }

    OodleXLog_Printf_v1("Oodle %s example_lz_chart <file>\n",OodleVersion);
    
    if ( argc < 2 )
    {
        fprintf(stderr,"error: specify a sample data file to test on.\n");
        return 10;
    }
    
    const char * in_name = argv[1];
    OodleXLog_Printf_v1("lz_chart loading %s...\n",in_name);

    // read the input file to the global buffer :
    OO_S64 in_size_64;
    void * in_buffer = OodleXIOQ_ReadMallocWholeFile_AsyncAndWait(in_name,&in_size_64);
    
    if ( ! in_buffer)
    {
        OodleXLog_Printf_v0("failed to read %s\n",in_name); 
        return 10;
    }
    
    OodleXLog_Printf_v1("file size : " OOEX_S64_FMT "\n",in_size_64);
    
    OO_SINTa in_size = OodleX_S64_to_SINTa_check( in_size_64 );
        
    //-----------------------------------------------------
    
    // test parameters :
    //  increase these to get more reliable timing
    //  decrease these to run faster
    int min_encode_repeats = 2;
    int min_decode_repeats = 5;
    double min_total_seconds = 2.0;
    
    #if 1
    
    // test set of compressors : 
    OodleLZ_Compressor chart_compressors[] = {
        OodleLZ_Compressor_Selkie, OodleLZ_Compressor_Mermaid, OodleLZ_Compressor_Kraken, OodleLZ_Compressor_Leviathan
    };
        
    #define EXAMPLE_LZ_CHART_NUM_COMPRESSORS    ((int)(sizeof(chart_compressors)/sizeof(chart_compressors[0])))
    OOEX_ASSERT( EXAMPLE_LZ_CHART_NUM_COMPRESSORS == 4 );
    
    #define EXAMPLE_LZ_CHART_NUM_LEVELS 5
    
    OodleLZ_CompressionLevel chart_levels1[] = {
        OodleLZ_CompressionLevel_HyperFast4, OodleLZ_CompressionLevel_HyperFast3, OodleLZ_CompressionLevel_HyperFast2, OodleLZ_CompressionLevel_HyperFast1, OodleLZ_CompressionLevel_SuperFast,
    };
    OodleLZ_CompressionLevel chart_levels2[] = {
        OodleLZ_CompressionLevel_VeryFast, OodleLZ_CompressionLevel_Fast, OodleLZ_CompressionLevel_Normal, OodleLZ_CompressionLevel_Optimal1, OodleLZ_CompressionLevel_Optimal3
    };
    OOEX_ASSERT( ((int)(sizeof(chart_levels1)/sizeof(chart_levels1[0]))) == EXAMPLE_LZ_CHART_NUM_LEVELS );
    OOEX_ASSERT( ((int)(sizeof(chart_levels2)/sizeof(chart_levels2[0]))) == EXAMPLE_LZ_CHART_NUM_LEVELS );
    
    #else
    
    // test just 1 :
    
    OodleLZ_Compressor chart_compressors[] = {
        OodleLZ_Compressor_Kraken
    };
        
    #define EXAMPLE_LZ_CHART_NUM_COMPRESSORS    1
    
    #define EXAMPLE_LZ_CHART_NUM_LEVELS 1
    
    OodleLZ_CompressionLevel chart_levels1[] = {
        OodleLZ_CompressionLevel_None
    };
    OodleLZ_CompressionLevel chart_levels2[] = {
        OodleLZ_CompressionLevel_Optimal2
    };
    
    #endif
    
    //===========================================================

    OO_SINTa comp_lens[EXAMPLE_LZ_CHART_NUM_COMPRESSORS][EXAMPLE_LZ_CHART_NUM_LEVELS];
    double decode_speeds[EXAMPLE_LZ_CHART_NUM_COMPRESSORS][EXAMPLE_LZ_CHART_NUM_LEVELS];
    double encode_speeds[EXAMPLE_LZ_CHART_NUM_COMPRESSORS][EXAMPLE_LZ_CHART_NUM_LEVELS];
    
    //-----------------------------------------------------
    
    void * comp_buf;

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Invalid,in_size);

    comp_buf = OodleXMallocBig(comp_buf_size);
        
    void * decode_buffer = OodleXMallocBig( in_size );
        
    //-----------------------------------------------------
    // allocate scratch 

    // get enough so the decoder won't allocate
    //  and encoders up to Leviathan-Normal won't either
    //  but optimals will
        
    s_scratch_memory_size = OodleLZ_GetCompressScratchMemBound(OodleLZ_Compressor_Leviathan,OodleLZ_CompressionLevel_Normal,in_size,NULL);
    
    s_scratch_memory_size = OOEX_MAX( s_scratch_memory_size , OodleLZDecoder_MemorySizeNeeded() );
    
    s_scratch_memory = OodleXMalloc(s_scratch_memory_size);

    //-----------------------------------------------------
    
    bar();
    OodleXLog_Printf_v1("Selkie : super fast to encode & decode, least compression\n");
    OodleXLog_Printf_v1("Mermaid: fast decode with better-than-zlib compression\n");
    OodleXLog_Printf_v1("Kraken : good compression, fast decoding, great tradeoff!\n");
    OodleXLog_Printf_v1("Leviathan : very high compression, slowest decode\n");
    bar();
    OodleXLog_Printf_v1("chart cell shows | raw/comp ratio : encode MB/s : decode MB/s | \n");
    OodleXLog_Printf_v1("All compressors run at various encoder effort levels (SuperFast - Optimal).\n");
    OodleXLog_Printf_v1("Many repetitions are run for accurate timing.\n");
    bar();
    
    for(int twice=0;twice<2;twice++)
    {
        const OodleLZ_CompressionLevel * chart_levels =
            twice ? chart_levels2 : chart_levels1;
    
        int num_compressors = EXAMPLE_LZ_CHART_NUM_COMPRESSORS;
        // don't bother running Leviathan at the HyperFast modes :
        if ( ! twice && chart_compressors[num_compressors-1] == OodleLZ_Compressor_Leviathan ) num_compressors--;
    
        header(chart_levels,EXAMPLE_LZ_CHART_NUM_LEVELS);
        
        for(int c=0;c<num_compressors;c++)
        {
            OodleLZ_Compressor compressor = chart_compressors[c];
            
            OodleXLog_Printf_v1("%s|",truncated_compressor_name(compressor));
            
            // iterate backwards so we start the slowest first
            for(int l=0;l<EXAMPLE_LZ_CHART_NUM_LEVELS;l++)
            {           
                OodleLZ_CompressionLevel level = chart_levels[l];
            
                time_and_len tl = 
                    Encode_And_Time(in_buffer,in_size,comp_buf,compressor,level,NULL,min_encode_repeats,min_total_seconds);
                
                OO_SINTa comp_len = tl.len;
                double encode_seconds = tl.time;
                
                double encode_mbps = (in_size/1000000.0) / encode_seconds;      
                encode_speeds[c][l] = encode_mbps;
                
                comp_lens[c][l] = comp_len;
                
                double ratio = (double)in_size/comp_len;
                
                if ( ratio >= 10.0 )
                    OodleXLog_Printf_v1("%4.1f",ratio);
                else
                    OodleXLog_Printf_v1("%4.2f",ratio);
                
                if ( encode_mbps >= 10.0 )
                    OodleXLog_Printf_v1(":%3d:",(int)(encode_mbps+0.5));
                else
                    OodleXLog_Printf_v1(":%3.1f:",encode_mbps);
                    
                double decode_seconds = Decode_And_Time( comp_buf, comp_len, decode_buffer, in_size , 
                    min_decode_repeats,min_total_seconds);
        
                OOEX_ASSERT_ALWAYS( memcmp(decode_buffer,in_buffer,in_size) == 0 );
                        
                double decode_mbps = (in_size/1000000.0) / decode_seconds;              
                decode_speeds[c][l] = decode_mbps;
                
                OodleXLog_Printf_v1("%4d|",(int)(decode_mbps+0.5));
            }
            OodleXLog_Printf_v1("\n");
            
        }

        //-----------------------------------------------------

        bar();  
        OodleXLog_Printf_v1("compression ratio (raw/comp):\n");
        header(chart_levels,EXAMPLE_LZ_CHART_NUM_LEVELS);
        
        for(int c=0;c<num_compressors;c++)
        {
            OodleLZ_Compressor compressor = chart_compressors[c];
            
            OodleXLog_Printf_v1("%s|",truncated_compressor_name(compressor));
            
            // iterate backwards so we start the slowest first
            for(int l=0;l<EXAMPLE_LZ_CHART_NUM_LEVELS;l++)
            {           
                double ratio = (double)in_size/comp_lens[c][l];
                
                OodleXLog_Printf_v1("%9.3f    |",ratio);
            }
            OodleXLog_Printf_v1("\n");
        }

        bar();
        OodleXLog_Printf_v1("encode speed (MB/s):\n");
        header(chart_levels,EXAMPLE_LZ_CHART_NUM_LEVELS);
        
        for(int c=0;c<num_compressors;c++)
        {
            OodleLZ_Compressor compressor = chart_compressors[c];
            
            OodleXLog_Printf_v1("%s|",truncated_compressor_name(compressor));
            
            // iterate backwards so we start the slowest first
            for(int l=0;l<EXAMPLE_LZ_CHART_NUM_LEVELS;l++)
            {           
                OodleXLog_Printf_v1("%11.3f  |",encode_speeds[c][l]);
            }
            OodleXLog_Printf_v1("\n");
        }

        bar();
        OodleXLog_Printf_v1("decode speed (MB/s):\n");
        header(chart_levels,EXAMPLE_LZ_CHART_NUM_LEVELS);
        
        for(int c=0;c<num_compressors;c++)
        {
            OodleLZ_Compressor compressor = chart_compressors[c];
            
            OodleXLog_Printf_v1("%s|",truncated_compressor_name(compressor));
            
            // iterate backwards so we start the slowest first
            for(int l=0;l<EXAMPLE_LZ_CHART_NUM_LEVELS;l++)
            {           
                OodleXLog_Printf_v1("%11.3f  |",decode_speeds[c][l]);
            }
            OodleXLog_Printf_v1("\n");
        }

        bar();

    } // twice

    //-----------------------------------------------------

    OodleXFree(s_scratch_memory);
    OodleXFreeBig(decode_buffer);
    OodleXFreeBig(comp_buf);        
    OodleXFree_IOAligned(in_buffer);

    //OodleX_Shutdown();
    OodleX_Shutdown(NULL,OodleX_Shutdown_LogLeaks_Yes,0);
    
    //OodleXLog_Printf_v1("press a key\n");
    //fgetc(stdin);
    
    return 0;
}

//=================================================

 
example_lz : Example demonstrating LZ compression and decompressionExamplesexample_lz_noallocs : Example demonstrating Oodle compression with no allocations

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_CopyFile_Async
Navigation
 OodleXIOQ_MakeAllDirs_Async
 OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_CopyFile_Async( const char * from,
                                       const char * to,
                                       OO_U32 oodleCopyFileFlags,
                                       OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                       OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                       const OodleXHandle * dependencies OODEFAULT( NULL ),
                                       OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to copy a file
Parameters
from  source file name (VFS,UTF8)
to  dest file name (VFS,UTF8)
oodleCopyFileFlags  bitwise OR of flags from OodleXCopyFileFlags
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Copy a file as a single IOQ op.

NOTE : you generally do not want this. Use Oodle_CopyFile_Async instead. Using this call blocks the IOQ from servicing streams or doing other work.

CopyFile is a single IOQ op so it is guaranteed to be done before a subsequent call to OodleIOQ_OpenForRead on the to file, so it is useful for async transparent mirroring. (the same is not true of Oodle_CopyFile_Async which has undefined scheduling).


 

OodleXIOQ_MakeAllDirs_AsyncOodleX low level async ioOodleXIOQ_ReadMallocWholeFile_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleIOQ
Navigation
 OodleX low level async io
 Welcome to Oodle
 Change Log

About OodleIOQ

OodleIOQ is the lowest level async IO layer that the client should use.

(OodleXFileOpsVTable is lower level; the IOQ uses it to do its file ops, but the client should not call OodleXFileOpsVTable functions directly)

Every file IO operation on an OodleIOQ file is completely asynchronous, they never block the calling thread. Calls like OodleXIOQ_CloseFile_Async just start an operation; you can then OodleX_Wait on the returned handle if you like. (there are convenience wrappers provided (like OodleXIOQ_WriteWholeFile_AsyncAndWait) which just start an async op and then wait on it).

An OodleXIOQFile does not have a file pointer (position) or buffer associated with it. They are stateless. Multiple threads may be using the same OodleXIOQFile at the same time.

Many people are initially confused by the fact that OodleXIOQ_OpenForRead_Async is asynchronous. That means that if you try to open a file that doesn't exist, the call to OodleXIOQ_OpenForRead_Async will still succeed ; the success of that call only means that it was successful in starting the operation. If you want to know if the operation actually succeeded, you must get the status of the OodleXHandle returned. OodleXIOQ_OpenForRead_Async will always create an OodleXIOQFile, but it may not contain a valid disk file until/unless the operation succeeds.

Every operation you start on the IOQ has a corresponding OodleXHandle. That handle can be used to check status (OodleX_GetStatus) or wait on completion. If you don't need to track the progress of an IO operation, you can make the handle have a autoDelete of OodleXHandleAutoDelete_Yes, and just ignore the handle.


IO Operation Ordering

IOQ operations are implicitly FIFO in the absense of other user-provided scheduling constraints. The IOQ may transparently reorder operations when that reordering cannot affect their behavior; for example Reads and OpensForRead can be moved against each other, but Reads will never be moved across Writes.

FIFO ordering is very handy for the client to batch up chains of operations and know what order they will be done in.

For example you can do

OodleXIOQ_Read_Async(file1,buffer,sizeof(buffer),0);

OodleXIOQ_Write_Async(file2,buffer,sizeof(buffer),0);

this will enqueue a read from file1 into "buffer" and then a write to file2 from "buffer". Because of the implicit FIFO order, this will do what you expect - the read will complete and fill buffer before I try to write from it. Without the implicit FIFO those operations would have to be explicitly ordered by dependencies.

Every OodleIOQ function that starts an async operation optionally takes a list of dependencies. If you pass dependencies, then that operation will not participate in the FIFO sequence; its ordering is set by the completion of the dependencies.


Alignment and Buffering

The Oodle IOQ can be used with aligned or unaligned IO. Unaligned IO is faster on some platforms, and sometimes has lower memory use, so it should be used when possible. Some platforms can only do true async IO on sector-aligned boundaries, so if you use unaligned IO on those platforms the IO thread will be running synchronous IO (it will still appear asynchronous to the client code using the IOQ).

If you don't want to use aligned IO, open all files with OodleXFileOpenFlags_Buffered. Then the file sizes and positions you pass to IOQ can be arbitrary.

If you wish to use aligned IO, all pointers and file positions that are used in OodleIOQ calls must be aligned. The easiest way is to align everything to OODLEX_IO_MAX_ALIGNMENT, which works on any platform. To do this you can allocate your buffers using OodleXMalloc_IOAligned (or you can always align pointers yourself).

If you don't want to use OODLEX_IO_MAX_ALIGNMENT, OodleXIOQ_GetInfo will report the actual alignment required for a given file on that platform. Note that the alignment may differ on different devices, so it should not be stored as a global.

To do non-aligned IO when you have opened your files in aligned mode, you have a few options. To read an unaligned part of a file, simply move the start position back to the previous alignment, with OodleX_IOAlignDownS64 and move the end position up with OodleX_IOAlignUpS64 , then read this larger block and you will find your unaligned piece inside. Alternatively, OodleXIOQ_ReadUnalignedAdjustPointer_Async is a helper which does this for you.

Another option is to use the higher level OodleFile routines, which do large aligned IO's to fill a buffer and let you grab arbitrary pieces of that buffer. You could also use an OodleIOQStream if you are always reading forward linearly.

To write an unaligned portion of a file (on a file opened in aligned-IO mode), it's probably best to use the higher level OodleFile interface which does synchronous buffering for you. Otherwise you have to first read the surrounding larger aligned chunks, replace your unaligned portion, and write it back out again.


Tracking Errors

You can track the error status and completion of each request using the OodleXHandle it returns.

Alternatively, OodleIOQ supports a more "file-first" mode of usage.

OodleXIOQFile tracks the last error of any request on it, so you can just fire off a bunch of requests and then check OodleXIOQ_GetLastError on the file to see if any of them failed.

OodleXIOQFile also tracks the last handle of any operation started on it, so you can use OodleIOQ_GetLastRequest or OodleXIOQ_WaitLastRequest to check if all ops on a file are done.

With this mode of usage, you probably want to start the operations with a autoDelete of OodleXHandleAutoDelete_Yes so that you don't have to delete them manually.


Threading Semantics

IOQ operations are all inherently atomic and thread safe. However, your user objects which you pass to the IOQ calls must be managed by you.

Buffers used in IOQ calls must be kept allocated for the lifetime of the operation. eg. if you call OodleXIOQ_Read_Async , the memory pointer you pass in must be kept allocated until the handle completes. Don't use memory on the stack if your function can return before the operation is complete.

Strings passed into IOQ requests (eg. file names) are copied and can be freed immediately after starting the operation.
 

OodleX low level async ioOodleX low level async ioOODLEX_IO_MAX_ALIGNMENT

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_JOB_NULL_HANDLE
Navigation
 OODLE_JOB_MAX_DEPENDENCIES
 t_fp_Oodle_Job
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_JOB_NULL_HANDLE (0)
Discussion
Value 0 of Jobify handles is reserved to mean none * Wait(OODLE_JOB_NULL_HANDLE) is a nop * if RunJob returns OODLE_JOB_NULL_HANDLE it means the job * was run synchronously and no wait is required
 
OODLE_JOB_MAX_DEPENDENCIESCore Baset_fp_Oodle_Job

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNet_Plugins_SetAssertion
Navigation
 OodleNet_Plugins_SetPrintf
 Welcome to Oodle
 Change Log
// Function prototype:
t_fp_OodleNet_Plugin_DisplayAssertion * OodleNet_Plugins_SetAssertion( t_fp_OodleNet_Plugin_DisplayAssertion * fp_rrDisplayAssertion );
Discussion
Install the callback used by Oodle Core for asserts
Parameters
fp_rrDisplayAssertion  function pointer to your assert display function
Return Value
return  returns the previous function pointer
Discussion

Use this function to install your own display for Oodle Core assertions. This will only happen in debug builds.

The default implementation in debug builds, if you install nothing, uses the C stderr printf for logging, except on Microsoft platforms where it uses OutputDebugString.

WARNING : this function is NOT thread safe! It should be done only once and done in a place where the caller can guarantee thread safety.


 

OodleNet_Plugins_SetPrintfNetwork pluginst_fp_OodleNet_Plugin_MallocAligned

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_ALLOW_DEPRECATED_COMPRESSORS
Navigation
 OODLELZ_LOCALDICTIONARYSIZE_MAX
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_ALLOW_DEPRECATED_COMPRESSORS 
Discussion
If you need to encode with the deprecated compressors, define this before including oodle2.h
Discussion
You may still decode with them without defining this.
 
OodleAPI_LZ_CompressorsOodleAPI_LZ_CompressorsOODLELZ_LOCALDICTIONARYSIZE_MAX

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Examples
Navigation
 Welcome to Oodle
 Change Log
  •  How to build and use the Oodle examples
  •  example_lz : Example demonstrating LZ compression and decompression
  •  example_lz_chart : Example that makes a chart of OodleLZ options
  •  example_lz_noallocs : Example demonstrating Oodle compression with no allocations
  •  example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
  •  example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
  •  example_lz_threadphased : Example of 2-thread ThreadPhased decoding
  •  example_network_client : Example with simple network client support
  •  example_packet : Example demonstrating network packet compression

 
  How to build and use the Oodle examples

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: What is SINTa? How do I load files bigger than 2 GB?
Navigation
 FAQ: I ran out of OodleHandle table slots; what do I do?
 Frequently Asked Questions
 FAQ: My Files aren't loading right and I can't track it down
 Welcome to Oodle
 Change Log

Oodle uses 64 bit integers for file sizes and file positions. This allows you to manipulate files that are larger than 2 GB (31 bits) even on 32 bit systems.

Oodle uses a variable called "SINTa" for most memory buffer sizes. The "a" stands for address and indicates it is an integer the size of an address (eg. 32 bits on 32 bit systems and 64 bits if pointers are 64 bits). It's the same idea as "intptr_t" or "ptrdiff_t" in standard C.

Many of the Oodle APIs, particularly the higher level ones that are provided for convenience, only work if they can load an entire file into memory (for example, things like OodleXIOQ_ReadMallocWholeFile_Async).

On 64 bit systems, everything is simple - SINTa is the same as S64 and you can make buffers big enough to load entire files.

On 32 bit systems, SINTa is S32, and not all file sizes can fit in an SINTa.

Whenever you have to cast an S64 to an SINTa, the safest way is to use OodleX_S64_to_SINTa_check ; on 64 bit systems this is a no-op, but on 32 bit systems it emits an error if the size is too large.

If you want to write code that can handle files larger than 2 GB on a 32 bit system, you can do this with Oodle, you simply have to use the lower level APIs and avoid the higher level helpers that deal with whole file buffers.

For example the basic OodleXIOQ_Read_Async that all other Oodle reads is built on can be used to read small parts of a large file.

Similarly the whole buffer decompression routines like OodleLZ_Decompress cannot be used if the whole buffer doesn't fit in memory, but you can use the incremental decompression (OodleLZDecoder_DecodeSome) to stream a larger-than-memory file through memory and decompress it.

Oodle never makes match references further away than 1 GB, so you can encode huge buffers in 64-bit and they are gauranteed to be decodeable in 32-bit.

Oodle does not currently provide compression routines that can handle larger than memory buffers. But you can simply compress part of the file independently and concatenate together the chunks, because OodleLZ compressed data is binary concatenatable. (see About OodleLZ). For example you could encode 0.5 GB per call, with 0.5 GB of overlap to the previous chunk. The data you emit this way could be decoded in one call on 64 bit systems, or with multiple calls on 32 bit.

See the "oozi" example for compression of very large files.

NOTE : I highly recommend using Oodle in 64 bit. Not only will it avoid memory limitations, it is significantly faster.
 

FAQ: I ran out of OodleHandle table slots; what do I do?Frequently Asked QuestionsFAQ: My Files aren't loading right and I can't track it down

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_CloseFileRename_Async
Navigation
 OodleXIOQ_CloseFile_Async
 OodleXIOQ_Read_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_CloseFileRename_Async( OodleXIOQFile file,
                                              const char * renameTo,
                                              OO_S64 truncateFileSize OODEFAULT( OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE ),
                                              OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                              OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                              const OodleXHandle * dependencies OODEFAULT( NULL ),
                                              OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a close-file-rename request
Parameters
file  the file to close
renameTo  file to rename to (VFS, UTF-8)
truncateFileSize  (optional) truncate an OpenforWrite file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Does an OodleXIOQ_CloseFile_Async , then renames the file to "renameTo" - but only if there were no errors in writing the file. To stop unimportant errors from causing OodleXIOQ_CloseFileRename_Async to fail, use OodleXIOQ_ClearError before calling this.

renameTo can be NULL to cancel the close and delete the temp file.

Useful with OodleXIOQ_OpenForWriteTempName_Async.
 

OodleXIOQ_CloseFile_AsyncOodleX low level async ioOodleXIOQ_Read_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle ozip
Navigation
 Index of Abouts
 About Oodle
 About Oodle on Platforms
 Welcome to Oodle
 Change Log

ozip is utility for Oodle that's designed to act like gzip, for piping compresed streams on Unix-like platforms.

ozip comes with Oodle as a compiled executable (in the bin/ directory), and is also available as source code.

The ozip code is public domain and is provided on github :

https://github.com/jamesbloom/ozip/

Oodle is not public domain and cannot be used without an Oodle license. The compiled ozip executable with Oodle is not public domain.

ozip can be used to test Oodle without having to write any code.

ozip -k --kraken -6 lzt99

makes lzt99.ooz with Kraken at level 6 (Optimal2)

When working on pipes, ozip incrementally flushes data so the stream keeps moving and memory use is limited. It can be used as a drop in to most programs that expect gzip-like pipes. For example with tar :


 tar -cf archive.tar.ooz -I 'ozip --fast' subdir

 tar cf - subdir | ozip --best > archive.tar.gz

ozip can also be used for benchmarking. In benchmark mode it makes no output and runs the operation many times to get accurate timing. You can use this to benchmark specific files with specific options, unlike example_lz_chart : Example that makes a chart of OodleLZ options which runs a matrix of compressors and levels with default options.

For example :


ozip -b lzt02
K 4 lzt02            :    755121 ->    161674 (4.671),  57.6 MB/s, 1808.7 MB/s

ozip -b lzt02 -9 --kraken
K 9 lzt02            :    755121 ->    135523 (5.572),   1.9 MB/s, 1547.4 MB/s

ozip -b lzt02 -9 --hydra
H 9 lzt02            :    755121 ->    130035 (5.807),   0.6 MB/s, 1291.0 MB/s

ozip -b lzt02 -9 --hydra -os512
H 9 lzt02            :    755121 ->    141414 (5.340),   0.6 MB/s, 1818.3 MB/s

zstd -b22 lzt02
22#lzt02             :    755121 ->    197827 (3.817), 10.21 MB/s , 815.0 MB/s

See some tips on benchmarking here : Tips for benchmarking a compressor
 

Index of AboutsAbout OodleAbout Oodle on Platforms

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_ReadAndDecompress_Wide_Async
Navigation
 OodleXLZ_Decompress_Narrow_Async
 OodleXLZ_Decompress_Wide_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_ReadAndDecompress_Wide_Async( OO_U32 asyncSelect,
                                                    const OodleLZ_SeekTable * seekTable,
                                                    const void * packedDataPtr,
                                                    OO_SINTa packedLen,
                                                    OO_SINTa packedLenPreviouslyRead,
                                                    OodleXIOQFile packedFile,
                                                    OO_S64 packedDataStartPos,
                                                    void * rawArray,
                                                    OO_SINTa rawArrayLen,
                                                    OodleLZ_FuzzSafe fuzzSafe,
                                                    OodleLZ_CheckCRC checkCRC,
                                                    OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                                                    void * decBufBase OODEFAULT( NULL ),
                                                    OO_SINTa decBufSize OODEFAULT( 0 ),
                                                    OodleLZ_PackedRawOverlap packedRawOverlap OODEFAULT( OodleLZ_PackedRawOverlap_No ),
                                                    OodleXIOQFile writeToFile OODEFAULT( 0 ),
                                                    OO_S64 writeToFileStartPos OODEFAULT( 0 ),
                                                    OodleXHandle * pWriteHandleGroup OODEFAULT( 0 ),
                                                    OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                    const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                    OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an async LZ decompress, possibly read packed data and write raw data
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run
seekTable  seek locations as created by OodleLZ_CreateSeekTable
packedDataPtr  pointer to LZ compressed data
packedLen  compressed data length
packedLenPreviouslyRead  number of packed bytes already in packedDataPtr from previous IO ; eg. packedLen if the whole buffer is full
packedFile  OodleXIOQFile to read packed bytes from
packedDataStartPos  file position where the packed data starts (must be misaligned the same way as packedDataPtr)
rawArray  pointer to memory filled with decompressed data
rawArrayLen  length of decompressed data
checkCRC  if OodleLZ_CheckCRC_Yes, the decompressor checks the crc to ensure data integrity
verbosity  (optional) if not OodleLZ_Verbosity_None, will log some information
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder.
decBufSize  (optional) size of circular buffer starting at decBufBase
packedRawOverlap  (optional) if OodleLZ_PackedRawOverlap_Yes, the compressed data is in the same memory array as the output raw data
writeToFile  (optional) OodleXIOQFile to write raw data to
writeToFileStartPos  (optional) file position where writeToFile should start (must be OODLEX_IO_MAX_ALIGNMENT aligned)
pWriteHandleGroup  (optional) if writeToFile is given, this is filled with an OodleAsyncGroup OodleXHandle containing all the file IO operations
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  OodleXHandle to the operation, or OodleXHandle_Null for invalid arguments
Discussion

Start an async LZ decompress with the runner specified in asyncSelect.

The entire Read-Decomp-Write is done with maximum parallelism; reads are done in chunks, as each compressed chunk is available, it is decompressed, and as raw chunks are done, they are written.

A note on the alignment of packedDataPtr and packedDataStartPos : the simplest way is if both are OODLEX_IO_MAX_ALIGNMENT. However, if the packed data starts some non-aligned way into the file, ensure the misalignment of both is the same. This is automatic if you allocate a buffer to correspond to the whole file, or start your read at the preceding aligned position.

If you have the data already read into memory, use OodleXLZ_Decompress_Wide_Async instead.

To use OodleLZ_PackedRawOverlap_Yes , make a buffer of size at least OodleLZ_GetInPlaceDecodeBufferSize ; you then read the compressed data in to the end of that array, and decompress with the raw pointer set to the front of that array. This lets you avoid allocating two large arrays. It does hurt parallelism.

rawArray and packedDataPtr memory blocks passed to this function must be kept alive for the duration of the async.

To use this function, you should have stored the seekTable for the compressed data in a file.

NOTE !! : if writeToFile is provided, the writes are async and are NOT necessarily done when the returned handle is done; the returned handle is for the decompress. They are done when the handle in pWriteHandleGroup is done. You must not free the buffer being written until *pWriteHandleGroup is done.
 

OodleXLZ_Decompress_Narrow_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Decompress_Wide_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_SetDefaultFileOps
Navigation
 OodleX_GetDefaultFileOps
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_SetDefaultFileOps( const OodleXFileOpsVTable * pNewVTable );
Discussion
Change the default file ops vtable
Discussion
pNewVTable is copied into the global default file ops.

These file ops are used by Oodle whenever no other vtable is provided.

WARNING : access to OodleXFileOpsVTable is not thread safe. It should generally only be done at app initialization time to set your desired func pointers, and then not done thereafter.
 

OodleX_GetDefaultFileOpsOodleX low level async ioOodleXIOQFile

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetDecodeBufferSize
Navigation
 OodleLZ_GetCompressedBufferSizeNeeded
 OodleLZ_GetInPlaceDecodeBufferSize
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetDecodeBufferSize( OodleLZ_Compressor compressor,
                                      OO_SINTa rawSize,
                                      OO_BOOL corruptionPossible );
Discussion
Get the size you must malloc the decode (raw) buffer
Parameters
compressor  compressor used; OodleLZ_Compressor_Invalid to make it enough for any compressor
rawSize  uncompressed (raw) size without padding
corruptionPossible  true if it is possible for the decoder to get corrupted data
Return Value
return  size of buffer to malloc; slightly larger than rawSize if padding is needed
Discussion

As of Oodle 2.9.0 this function is deprecated. For all new codecs you can just use the size of the uncompressed data for the decode buffer size (rawSize), no padding is needed.

Note that LZB16 is still supported in 2.9.0 but does require padding when used in a circular window (which is deprecated).

This padding is necessary for the older compressors when FuzzSafe_No is used. The old compressors and FuzzSafe_No are no longer supported.

If corruptionPossible is true, a slightly larger buffer size is returned.

If corruptionPossible is false, then you must ensure that the decoder does not get corrupted data, either by passing OodleLZ_CheckCRC_Yes , or by your own mechanism.

Note about possible overrun in LZ decoding (applies to the old non-fuzz-safe compressors) : as long as the compresseddata is not corrupted, and you decode either the entire compressed buffer, or an integer number of "seek chunks" (OODLELZ_BLOCK_LEN), then there will be no overrun. So you can decode LZ data in place and it won't stomp any following bytes. If those conditions are not true (eg. decoding only part of a larger compressed stream, decoding around a circular window, decoding data that may be corrupted), then there may be some limited amount of overrun on decode, as returned by OodleLZ_GetDecodeBufferSize.


 

OodleLZ_GetCompressedBufferSizeNeededOodleAPI_LZ_CompressorsOodleLZ_GetInPlaceDecodeBufferSize

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignDownS32
Navigation
 OodleX_IOAlignUpSINTa
 OodleX_IOAlignDownS64
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleX_IOAlignDownS32( const OO_S32 x );
Discussion
Align down to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x down to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_IOAlignUpSINTaOodleX UtilsOodleX_IOAlignDownS64

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_WaitJob
Navigation
 t_fp_OodleCore_Plugin_RunJob
 t_fp_OodleCore_Plugin_Printf
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleCore_Plugin_WaitJob )( OO_U64 job_handle,
                void * user_ptr );
Discussion
Function pointer type for OodleCore_Plugins_SetJobSystem
Parameters
job_handle  a job handle returned from RunJob. Never 0.
user_ptr  is passed through from the OodleLZ_CompressOptions.
Discussion

Waits until the job specified by job_handle is done and cleans up any associated resources. Oodle will call WaitJob exactly once for every RunJob call that didn't return 0.

If job_handle was already completed, this should clean it up without waiting.

A handle value should not be reused by another RunJob until WaitJob has been done with that value.

WaitJob will not be called from running jobs. It will be only be called from the original thread that invoked Oodle. If you are running Oodle from a worker thread, ensure that that thread is allowed to wait on other job threads.

See About Oodle Job Threading Plugins
 

t_fp_OodleCore_Plugin_RunJobCore pluginst_fp_OodleCore_Plugin_Printf

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXPriority
Navigation
 OodleXAsyncSelect
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXPriority
{
    OodleXPriority_Normal = 2,
    OodleXPriority_Default = OodleXPriority_Normal,
    OodleXPriority_NoPopOnWait = 1,
    OodleXPriority_Force32 = 0x40000000
};
Discussion
Priority for async tasks. Higher prority is done before lower priority. Async work is (on average) FIFO within priority group.
Enumerants
OodleXPriority_Normal  default priority ; does pop-on-wait
OodleXPriority_Default 
OodleXPriority_NoPopOnWait  for "high level" jobs
OodleXPriority_Force32 
Discussion
Core Jobs from Jobify are above priority Normal.

"low priority" work is "high level", large groups of actions "high priority" work is "low level", splitting small actions into micro actions that should complete soon


 

OODLEX_ASYNC_HANDLE_ERROROodleX async handle operationsOodleXAsyncSelect

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleAPI_OodleNetwork1
Navigation
 Oodle Network compression
 About Oodle Network Compression
 Welcome to Oodle
 Change Log

OodleNetwork1 compressor for shared-dictionary network channel compression.

  •  About OodleNetwork1
  •  Defines
    •  OODLENETWORK1_MAX_DICTIONARY_SIZE
    •  OODLENETWORK1_HASH_BITS_DEFAULT
    •  OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN
  •  Structures
    •  OodleNetwork1_Shared
    •  OodleNetwork1TCP_State
    •  OodleNetwork1UDP_State
    •  OodleNetwork1UDP_StateCompacted
  •  Functions
    •  OodleNetwork1_Shared_Size
    •  OodleNetwork1TCP_State_Size
    •  OodleNetwork1_CompressedBufferSizeNeeded
    •  OodleNetwork1_Shared_SetWindow
    •  OodleNetwork1TCP_State_Reset
    •  OodleNetwork1TCP_State_InitAsCopy
    •  OodleNetwork1TCP_Train
    •  OodleNetwork1TCP_Encode
    •  OodleNetwork1TCP_Decode
    •  OodleNetwork1UDP_Train
    •  OodleNetwork1UDP_State_Size
    •  OodleNetwork1UDP_Encode
    •  OodleNetwork1UDP_Decode
    •  OodleNetwork1UDP_StateCompacted_MaxSize
    •  OodleNetwork1UDP_State_Compact_ForVersion
    •  OodleNetwork1UDP_State_Compact
    •  OodleNetwork1UDP_State_Uncompact_ForVersion
    •  OodleNetwork1UDP_State_Uncompact
    •  OodleNetwork1_SelectDictionarySupported
    •  OodleNetwork1_SelectDictionaryFromPackets
    •  OodleNetwork1_SelectDictionaryFromPackets_Trials

 
Oodle Network compressionOodle Network compressionAbout OodleNetwork1

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleDecompressCallback
Navigation
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC OodleDecompressCallbackRet( OODLE_CALLBACK OodleDecompressCallback )( void * userdata,
                                      const OO_U8 * rawBuf,
                                      OO_SINTa rawLen,
                                      const OO_U8 * compBuf,
                                      OO_SINTa compBufferSize,
                                      OO_SINTa rawDone,
                                      OO_SINTa compUsed );
Discussion
User-provided callback for decompression
Parameters
userdata  the data you passed for pcbData
rawBuf  the decompressed buffer
rawLen  the total decompressed length
compBuf  the compressed buffer
compBufferSize  the total compressed length
rawDone  number of bytes in rawBuf decompressed so far
compUsed  number of bytes in compBuf consumed so far
Discussion

OodleDecompressCallback is called incrementally during decompression.
 

OodleLZ_GetChunkCompressorOodleAPI_LZ_Compressors 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
Navigation
 example_lz_overlap : Example demonstrating parallel overlap with OodleLZ
 Examples
 example_lz_threadphased : Example of 2-thread ThreadPhased decoding
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz_simple

Very simple example of OodleLZ memory -> memory compression & decompression.

Uses stdio for file IO to load an input file.

See example_lz : Example demonstrating LZ compression and decompression for more advanced OodleLZ usage.

example_lz_simple only uses Oodle Core, no Oodle Ext

include oodle2.h and not oodle2x.h


#include "../include/oodle2.h"

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz_simple
#endif

#include "read_whole_file.h"

extern "C" int main(int argc,char *argv[])
{
No initialization is needed for Oodle2 Core

we let Oodle Core use the default system plugins (the C stdlib) To change them, use Core plugins


    // optional check to make sure header matches lib :
    if ( ! Oodle_CheckVersion(OODLE_HEADER_VERSION) )
    {
        fprintf(stderr,"Oodle header version mismatch\n");
        return 10;
    }

    // get args :
    const char * in_name;
    if ( argc < 2 )
    {
        in_name = "r:\\testsets\\lztestset\\lzt02";
    }
    else
    {
        in_name = argv[1];
    }
    
read input file using stdio
    OO_SINTa length;
    void * buf = read_whole_file(in_name,&length);
    if ( ! buf )
    {
        fprintf(stderr,"couldn't open : %s\n",in_name);
        return 10;
    }
    

Run OodleLZ_Compress from memory (buf) to memory (compbuf)

Use the OodleLZ_Compressor_Kraken compressor. Kraken is an amazing balance of good compression and fast decode speed. It should generally be your first choice, then try Mermaid or Leviathan if you want faster decodes or more compression.

Use OodleLZ_CompressionLevel_Normal level of effort in the encoder. Normal is a balance of encode speed and compression ratio. Different levels trade off faster or slower encoding for compressed size. See OodleLZ_CompressionLevel.

See About OodleLZ for information on selection of the compression options.

This call is synchronous and not threaded; see example_lz : Example demonstrating LZ compression and decompression for an example using the async compression APIs.


    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Normal;
    //OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Optimal; // for high compression, slower encode

    // allocate memory big enough for compressed data :
    void * compbuf = malloc( OodleLZ_GetCompressedBufferSizeNeeded(compressor,length) + sizeof(length) );
    if ( compbuf == NULL )
        return 10;
        
    char * compptr = (char *)compbuf;
    memcpy(compptr,&length,sizeof(length));
    compptr += sizeof(length);

    // compress :
    OO_SINTa complen = OodleLZ_Compress(compressor,buf,length,compptr,level);
    compptr += complen;

    // log about it :
    //  full compressed size also includes the header +(int)sizeof(length)
    printf("%s compressed %d -> %d\n",in_name,(int)length,(int)complen);

Run OodleLZ_Decompress from memory (compbuf) to memory (decbuf)

Note that you must provide the exact decompressed size. OodleLZ data is headerless; store the size in your own header.

We will allocate the needed decoder scratch mem to pass in, then OodleLZ_Decompress will do no allocations internally. In real use you might want to keep the scratch mem allocated across calls so it doesn't need to be allocated and freed each time. The scratch mem can be reused, but cannot be used by multiple threads at the same time. See also example_lz_noallocs


    OO_SINTa declength;
    compptr = (char *)compbuf;
    memcpy(&declength,compptr,sizeof(declength));
    compptr += sizeof(declength);
    assert( length == declength );

    // malloc for decompressed buffer  :
    void * decbuf = malloc( declength );

    // allocate the decoder scratch memory needed
    //  if we pass NULL for these (the default argument)
    //  then OodleLZ_Decompress will allocate them internally
    OO_SINTa decoderScratchMemSize = OodleLZDecoder_MemorySizeNeeded(compressor,-1);
    void * decoderScratchMem = malloc( decoderScratchMemSize );

    // do the decompress :
    OO_SINTa decompress_return = OodleLZ_Decompress(compptr,complen,decbuf,declength,
        OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,NULL,0,NULL,NULL,
        decoderScratchMem,decoderScratchMemSize);

    // check it was successful :
    assert( decompress_return == length );
    assert( memcmp(buf,decbuf,length) == 0 );

    if ( decompress_return != length )
        return 10;

    printf("decompressed successfully.\n");
    
And finish up. No shutdown is needed for Oodle2 Core.

    // free all the memory :
    free(buf);
    free(compbuf);
    free(decbuf);
    free(decoderScratchMem);

    return 0;
}


 
example_lz_overlap : Example demonstrating parallel overlap with OodleLZExamplesexample_lz_threadphased : Example of 2-thread ThreadPhased decoding

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_WaitAll
Navigation
 OodleX_Wait
 OodleX_WaitDoneAllPending
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleX_WaitAll( const OodleXHandle * handles,
                             OO_S32 count,
                             OodleXHandleDeleteIfDone deleteIfDone OODEFAULT( OodleXHandleDeleteIfDone_No ) );
Discussion
Block the calling thread until none of the provided handles are Pending
Parameters
handles  array of OodleXHandle weak reference
count  number of handles in array
deleteIfDone  if OodleXHandleDeleteIfDone_Yes, all handle will be deleted
Return Value
return  handle status
Discussion

Blocks until ALL handles are done. Returns OodleXStatus_Error if any of the handles in the array is done with status OodleXStatus_Error.
 

OodleX_WaitOodleX async handle operationsOodleX_WaitDoneAllPending

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_FuzzSafe
Navigation
 OodleLZ_Decode_ThreadPhase
 OodleLZSeekTable_Flags
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_FuzzSafe
{
    OodleLZ_FuzzSafe_No = 0,
    OodleLZ_FuzzSafe_Yes = 1
};
Discussion
OodleLZ_FuzzSafe (deprecated)
Discussion
About fuzz safety:

Fuzz Safe decodes will not crash on corrupt data. They may or may not return failure, and produce garbage output.

Fuzz safe decodes will not read out of bounds. They won't put data on the stack or previously in memory into the output buffer.

As of Oodle 2.9.0 all compressors supported are fuzzsafe, so OodleLZ_FuzzSafe_Yes should always be used and this enum is deprecated.


 

OodleLZ_Decode_ThreadPhaseOodleAPI_LZ_CompressorsOodleLZSeekTable_Flags

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_SelectDictionaryFromPackets
Navigation
 OodleNetwork1_SelectDictionarySupported
 OodleNetwork1_SelectDictionaryFromPackets_Trials
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1_SelectDictionaryFromPackets( void * dictionary_to_fill,
                                                   OO_S32 dictionary_size,
                                                   OO_S32 htbits,
                                                   const void * * dictionary_packet_pointers,
                                                   const OO_S32 * dictionary_packet_sizes,
                                                   OO_S32 num_dictionary_packets,
                                                   const void * * test_packet_pointers,
                                                   const OO_S32 * test_packet_sizes,
                                                   OO_S32 num_test_packets );
Discussion
Build a dictionary for OodleNetwork1 from packets
Parameters
dictionary_to_fill  must be allocated by you to dictionary_size bytes; filled out
dictionary_size  size of dictionary_to_fill (should be <= OODLENETWORK1_MAX_DICTIONARY_SIZE)
htbits  hash table bits; should be the same value you pass to OodleNetwork1_Shared_SetWindow
dictionary_packet_pointers  pointers to packet data, num_dictionary_packets of them
dictionary_packet_sizes  packet sizes, num_dictionary_packets of them
num_dictionary_packets  number of packets used to make the dictionary
test_packet_pointers  pointers to packet data, num_test_packets of them
test_packet_sizes  packet sizes, num_test_packets of them
num_test_packets  number of packets used to test the dictionary
Return Value
return  true on success, false otherwise.
Discussion

Fills out a dictionary from packets by rating the packets and choosing the most useful.

Can be used with OodleNetwork1 UDP or TCP.

The dictionary_packet_pointers are used to fill the dictionary. The test_packet_pointers are used to evaluate the quality of the dictionary. They should not be the same packets! They should both be independent random representative samples of your network traffic.

The packets used here should not be the same ones used for training! (eg. with OodleNetwork1UDP_Train or OodleNetwork1TCP_Train).

This function is to be called before OodleNetwork1_Shared_SetWindow.

This function may take a lot of time if the test_packet_pointers set is too large. It should be large enough to provide good test ratings, but too large slows it down for no reason. Something like 50 MB of packet data is usually sufficient.

See TestOodleNetwork1PacketCoder_SelectDictionaryAndTrain in example_packet : Example demonstrating network packet compression

On platforms where OodleNetwork1_SelectDictionarySupported returns false, this function is not supported.

It's advised to run this function in 64-bit, as it can use a lot of memory.
 

OodleNetwork1_SelectDictionarySupportedOodleAPI_OodleNetwork1OodleNetwork1_SelectDictionaryFromPackets_Trials

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_FAILED
Navigation
 OODLELZ_QUANTUM_LEN
 OODLELZ_SCRATCH_MEM_NO_BOUND
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_FAILED (0)
Discussion
Return value of OodleLZ_Decompress on failure
 
OODLELZ_QUANTUM_LENOodleAPI_LZ_CompressorsOODLELZ_SCRATCH_MEM_NO_BOUND

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder
Navigation
 OodleLZ_DecodeSome_Out
 OodleLZ_SeekTable
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleLZDecoder;
Discussion
Opaque type for OodleLZDecoder
Discussion
See OodleLZDecoder_Create
 
OodleLZ_DecodeSome_OutOodleAPI_LZ_CompressorsOodleLZ_SeekTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_LogError
Navigation
 OodleXIOQ_GetErrorEnum
 OodleXIOQ_Wait
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXIOQ_LogError( OO_U32 code,
                         OodleXIOQFile file,
                         const char * pName OODEFAULT( NULL ) );
Discussion
Logs an OS error code with a detailed text message
Parameters
code  the error code, eg. from OodleXIOQ_GetStatus
file  the file that the error occurred on (or 0 for unknown)
pName  (optional) a tag to log with the error
Discussion

Calls OodleXLog_Printf to output a detailed error, as created by OodleXIOQ_GetErrorDetails.
 

OodleXIOQ_GetErrorEnumOodleX low level async ioOodleXIOQ_Wait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_State_InitAsCopy
Navigation
 OodleNetwork1TCP_State_Reset
 OodleNetwork1TCP_Train
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNetwork1TCP_State_InitAsCopy( OodleNetwork1TCP_State * state,
                                        const OodleNetwork1TCP_State * from );
Discussion
Initialize a OodleNetwork1TCP_State as a copy of another state
Parameters
state  OodleNetwork1TCP_State to initialize
from  OodleNetwork1TCP_State that was previously filled, to copy from
Discussion

Resets state to a copy of from state.

Generally from was made with OodleNetwork1TCP_Train and then saved to disk so that it could be stored on both the client and server.
 

OodleNetwork1TCP_State_ResetOodleAPI_OodleNetwork1OodleNetwork1TCP_Train

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignUpSINTa
Navigation
 OodleX_IOAlignUpS64
 OodleX_IOAlignDownS32
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleX_IOAlignUpSINTa( const OO_SINTa x );
Discussion
Align up to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x up to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_IOAlignUpS64OodleX UtilsOodleX_IOAlignDownS32

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleKickDelayed
Navigation
 OodleXHandleAutoDelete
 OodleXHandleDeleteIfDone
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXHandleKickDelayed
{
    OodleXHandleKickDelayed_No = 0,
    OodleXHandleKickDelayed_Yes = 1,
    OodleXHandleKickDelayed_Force32 = 0x40000000
};
Discussion
Normally async tasks are run as soon as possible; sometimes when spawning many tasks, you might not want to let the thread switch immediately, so it can be better to fire several tasks with OodleXHandle_KickDelayed and then kick them all together. ("kick" means activate worker threads to do the tasks)
Enumerants
OodleXHandleKickDelayed_No  (default) run async immediately
OodleXHandleKickDelayed_Yes  wait until manually kicked
OodleXHandleKickDelayed_Force32 

 
OodleXHandleAutoDeleteOodleX async handle operationsOodleXHandleDeleteIfDone

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetInfoByName_GetResult
Navigation
 OodleXIOQ_GetInfoByName_Async
 OodleXIOQ_SetInfoByName_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXIOQ_GetInfoByName_GetResult( OodleXHandle req,
                                                OodleXHandleDeleteIfDone andDeleteIfDone,
                                                OodleXFileInfo * pInfo );
Discussion
Finish an asynchronous GetInfo request
Parameters
req  the handle returned by OodleXIOQ_GetInfoByName_Async
andDeleteIfDone  if true and the request is done, delete the request
pInfo  filled out with an OodleXFileInfo
Return Value
return  OodleXStatus of the request
Discussion

pInfo is filled if the OodleXStatus returned is OodleXStatus_Done. If the return value is something else, pInfo is untouched (eg. not invalidated!).

If the file does not exist and OodleFileNotFoundIsAnError_No was passed, this function will return OodleXStatus_Done but pInfo will be set to a OodleXFileInfo:size of OODLEX_FILE_SIZE_INVALID;
 

OodleXIOQ_GetInfoByName_AsyncOodleX low level async ioOodleXIOQ_SetInfoByName_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Platforms
Navigation
 About Oodle ozip
 About Oodle
 About Oodle Job Threading Plugins
 Welcome to Oodle
 Change Log

Oodle is as uniform as possible across platforms.

The contents of the distribution, and how to build and use it on that platorm is described here.

  •  About Oodle on Windows
    •  About Oodle on Windows UWP
  •  About Oodle on PS4
  •  About Oodle on Nintendo Switch
  •  About Oodle on Mac
  •  About Oodle on Xbox One
  •  About Oodle on Linux
  •  About Oodle on IOS
  •  About Oodle on Android
  •  About Oodle on WASM

 
About Oodle ozipAbout OodleAbout Oodle on Windows

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ReadUnalignedAdjustPointer_Async
Navigation
 OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
 OodleXIOQ_MakeAllDirs_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_ReadUnalignedAdjustPointer_Async( void * * pPtr,
                                                         OodleXIOQFile file,
                                                         void * memory,
                                                         OO_SINTa readSize,
                                                         OO_S64 position,
                                                         OO_SINTa memorySize,
                                                         OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                         OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                         const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                         OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a read request with unaligned position or size
Parameters
pPtr  filled with a pointer to the read memory; NULL if the read is impossible
file  the file to act on
memory  memory to read into (no alignment required)
readSize  number of bytes to read (no alignment required)
position  file position to start the read (no alignment required)
memorySize  the size of the buffer at "memory"
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

memorySize should be larger than size ; generally at least aligned up with OodleX_IOAlignUpS64.

OodleXIOQ_ReadUnalignedAdjustPointer_Async reads a larger chunk than [position,size] , aligning down the start and aligning up the end. It reads somewhere into [memory,memorySize]. The returned pointer is somewhere in memory and contains the bytes you wanted from _position.

If memorySize is not big enough, it returns NULL.
 

OodleXIOQ_OpenWriteWholeFileCloseTempName_AsyncOodleX low level async ioOodleXIOQ_MakeAllDirs_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_MakeAllDirs_Async
Navigation
 OodleXIOQ_ReadUnalignedAdjustPointer_Async
 OodleXIOQ_CopyFile_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_MakeAllDirs_Async( const char * name,
                                          OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                          OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                          const OodleXHandle * dependencies OODEFAULT( NULL ),
                                          OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to make all dirs in name
Parameters
name  name of the file to make dirs for (VFS, UTF-8)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Makes the dirs in name in sequence. name can be a file name, or a path with trailing path delim.

eg. if name is "a/b/c/d" then dir a is made, then b, then c, but not d.
 

OodleXIOQ_ReadUnalignedAdjustPointer_AsyncOodleX low level async ioOodleXIOQ_CopyFile_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleAPI_LZ_Compressors
Navigation
 About OodleLZ Hydra
 Core LZ compression
 Welcome to Oodle
 Change Log

OodleLZ low level synchronous memory to memory lossless data compression.

  •  Defines
    •  OODLE_ALLOW_DEPRECATED_COMPRESSORS
    •  OODLELZ_LOCALDICTIONARYSIZE_MAX
    •  OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
    •  OODLELZ_BLOCK_LEN
    •  OODLELZ_BLOCK_MAX_COMPLEN
    •  OODLELZ_QUANTUM_LEN
    •  OODLELZ_FAILED
    •  OODLELZ_SCRATCH_MEM_NO_BOUND
  •  Enumerants
    •  OodleLZ_Verbosity
    •  OodleLZ_Compressor
    •  OodleLZ_PackedRawOverlap
    •  OodleLZ_CheckCRC
    •  OodleLZ_Profile
    •  OodleDecompressCallbackRet
    •  OodleLZ_CompressionLevel
    •  OodleLZ_Jobify
    •  OodleLZ_Decode_ThreadPhase
    •  OodleLZ_FuzzSafe
    •  OodleLZSeekTable_Flags
    •  OodleLZ_CompressScratchMemBoundType
  •  Structures
    •  OodleLZ_CompressOptions
    •  OodleLZ_DecodeSome_Out
    •  OodleLZDecoder
    •  OodleLZ_SeekTable
  •  Functions
    •  OodleLZ_Compress
    •  OodleLZ_Decompress
    •  OodleLZDecoder_Create
    •  OodleLZDecoder_MemorySizeNeeded
    •  OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
    •  OodleLZDecoder_Destroy
    •  OodleLZDecoder_Reset
    •  OodleLZDecoder_DecodeSome
    •  OodleLZDecoder_MakeValidCircularWindowSize
    •  OodleLZ_MakeSeekChunkLen
    •  OodleLZ_GetNumSeekChunks
    •  OodleLZ_GetSeekTableMemorySizeNeeded
    •  OodleLZ_FillSeekTable
    •  OodleLZ_CreateSeekTable
    •  OodleLZ_FreeSeekTable
    •  OodleLZ_CheckSeekTableCRCs
    •  OodleLZ_FindSeekEntry
    •  OodleLZ_GetSeekEntryPackedPos
    •  OodleLZ_CompressionLevel_GetName
    •  OodleLZ_Compressor_GetName
    •  OodleLZ_Jobify_GetName
    •  OodleLZ_CompressOptions_GetDefault
    •  OodleLZ_CompressOptions_Validate
    •  OodleLZ_Compressor_UsesWholeBlockQuantum
    •  OodleLZ_Compressor_UsesLargeWindow
    •  OodleLZ_Compressor_CanDecodeInCircularWindow
    •  OodleLZ_Compressor_CanDecodeThreadPhased
    •  OodleLZ_Compressor_CanDecodeInPlace
    •  OodleLZ_Compressor_MustDecodeWithoutResets
    •  OodleLZ_Compressor_CanDecodeFuzzSafe
    •  OodleLZ_Compressor_RespectsDictionarySize
    •  OodleLZ_GetCompressScratchMemBound
    •  OodleLZ_GetCompressScratchMemBoundEx
    •  OodleLZ_GetCompressedBufferSizeNeeded
    •  OodleLZ_GetDecodeBufferSize
    •  OodleLZ_GetInPlaceDecodeBufferSize
    •  OodleLZ_GetCompressedStepForRawStep
    •  OodleLZ_GetAllChunksCompressor
    •  OodleLZ_GetFirstChunkCompressor
    •  OodleLZ_GetChunkCompressor
  •  Typedefs
    •  OodleDecompressCallback

 
About OodleLZ HydraCore LZ compressionOODLE_ALLOW_DEPRECATED_COMPRESSORS

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_SetConfigValues
Navigation
 Oodle_GetConfigValues
 Oodle_SetUsageWarnings
 Welcome to Oodle
 Change Log
// Function prototype:
void Oodle_SetConfigValues( const OodleConfigValues * ptr );
Discussion
Set OodleConfigValues
Parameters
ptr  your desired OodleConfigValues
Discussion

Sets the global OodleConfigValues from your struct.

You should call Oodle_GetConfigValues to fill the struct, then change the values you want to change, then call Oodle_SetConfigValues.

This should generally be done before doing anything with Oodle (eg. even before OodleX_Init). Changing OodleConfigValues after Oodle has started has undefined effects.
 

Oodle_GetConfigValuesCore BaseOodle_SetUsageWarnings

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Decompress_Narrow_Async
Navigation
 OodleXLZ_Decompress_ThreadPhased_Narrow_Async
 OodleXLZ_ReadAndDecompress_Wide_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_Decompress_Narrow_Async( OO_U32 asyncSelect,
                                               const void * packedDataPtr,
                                               OO_SINTa packedLen,
                                               void * rawPtr,
                                               OO_SINTa rawChunkLen,
                                               OodleLZ_FuzzSafe fuzzSafe OODEFAULT( OodleLZ_FuzzSafe_No ),
                                               OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                                               OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                                               void * decBufBase OODEFAULT( NULL ),
                                               OO_SINTa decBufSize OODEFAULT( 0 ),
                                               OodleDecompressCallback * pcb OODEFAULT( NULL ),
                                               void * pcbData OODEFAULT( NULL ),
                                               void * decMem OODEFAULT( NULL ),
                                               OO_SINTa decMemSize OODEFAULT( 0 ),
                                               OodleLZ_Decode_ThreadPhase threadPhase OODEFAULT( OodleLZ_Decode_Unthreaded ),
                                               OodleXIOQFile writeToFile OODEFAULT( 0 ),
                                               OO_S64 writeToFileStartPos OODEFAULT( 0 ),
                                               OodleXHandle writeHandleGroup OODEFAULT( 0 ),
                                               OO_S32 writeHandleGroupIndex OODEFAULT( 0 ),
                                               OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                               const OodleXHandle * dependencies OODEFAULT( NULL ),
                                               OO_S32 numDependencies OODEFAULT( 0 ),
                                               OodleXPriority work_priority OODEFAULT( OodleXPriority_Normal ) );
Discussion
Start an async LZ decompress
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run
packedDataPtr  pointer to LZ compressed data
packedLen  compressed data length
rawPtr  pointer to memory filled with decompressed data
rawChunkLen  length of decompressed data
checkCRC  if OodleLZ_CheckCRC_Yes, the decompressor checks the crc to ensure data integrity
verbosity  (optional) if not OodleLZ_Verbosity_None, will log some information
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder.
decBufSize  (optional) size of circular buffer starting at decBufBase
pcb  (optional) OodleDecompressCallback called during decompression
pcbData  (optional) user data passed to pcb
writeToFile  (optional) OodleXIOQFile to write raw data to
writeToFileStartPos  (optional) file position where writeToFile should start (must be OODLEX_IO_MAX_ALIGNMENT aligned)
writeHandleGroup  (optional) OodleAsyncGroup handle which the write handle is put into
writeHandleGroupIndex  (optional) index in writeHandleGroup to use; must previously be OODLEX_ASYNC_HANDLE_PENDING
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Discussion

:return OodleXHandle to the operation, or OodleXHandle_Null for invalid arguments

Start an async LZ decompress with the runner specified in asyncSelect.

A Narrow decompress means the entire decompression is done on one thread. Data will always be decompressed sequentially, eg. in order.

rawChunkLen can be less than the entire original block, if it is a multiple of OODLELZ_BLOCK_LEN.

If provided, the OodleDecompressCallback is called as quanta of raw data are available. The callback may be called more often than OODLELZ_BLOCK_LEN granularity.

rawPtr and packedDataPtr memory blocks passed to this function must be kept alive for the duration of the async.

NOTE !! : if writeToFile is provided, the writes are async and are NOT necessarily done when the returned handle is done; the returned handle is for the decompress. The handle for the write can be retrieved by passing in writeHandleGroup. You must not free the buffer being written until the write operation is done.
 

OodleXLZ_Decompress_ThreadPhased_Narrow_AsyncOodleXAPI_LZ_AsyncOodleXLZ_ReadAndDecompress_Wide_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle2 Core vs Oodle2 Ext
Navigation
 Welcome to Oodle
 Change Log

Oodle Data Compression comes in two libraries on most platforms : Core and Ext. (On some embedded and mobile platforms, Ext is not provided)

(note Oodle Network is in its own SDK and libraries; see Oodle Network and Data SDK separation)

Oodle2 Core lib provides simple memory->memory synchronous operations. (eg. OodleLZ_Compress)

Oodle2 Ext extends the Core library by adding asynchronous and multi-threaded compression, as well as file IO and job scheduling. (eg. OodleXLZ_ReadAndDecompress_Stream_Async)

Core lib APIs start with Oodle_ , Ext lib APIs start with OodleX_

In general, it's recommended that you use OodleX in your tool chain, and use only Oodle Core in your shipping runtime.

OodleX provides its own allocator. Oodle Core defaults to using clib for allocations if needed, or you can replace that with Core plugins.

OodleX must be Init and Shutdown (see OodleX Startup and Shutdown) once in your app. Oodle Core does not require any initialization or shutdown, you can start calling Core APIs at any time (but not if you use OodleX! If you use OodleX, then Init must happen before any Oodle call, including any Core call).

Oodle Core is available on all platforms. Oodle Ext is not available on iOS or Android.

Oodle Core is entirely thread-safe and reentrant with no blocking primitives, because there is no global shared state.

As much as possible, the Oodle Core functions are "pure". That is, they act only on their arguments and do only CPU work, they do not interact with the system in any other way. The main exception is that Core can make calls to logging and allocation through user-provided system plugins (see Core plugins). It is possible to make all those plugins NULL in your shipping build; see FAQ: How do I use Oodle with no allocator?.

NOTE : on Windows DLL (and other DLL builds) do not import both Ext and Core! The Ext DLL contains Core as well.

NOTE : Ext automatically plugs itself into Core. You do not need to use the Plugin functions to install Ext to Core. That is not true of the new split libs Oodle Network and Oodle Texture. If you want to install OodleX to Net or Texture you have to call the Plugin functions explicitly.
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleCore_Plugins_SetAssertion
Navigation
 OodleCore_Plugins_SetPrintf
 Welcome to Oodle
 Change Log
// Function prototype:
t_fp_OodleCore_Plugin_DisplayAssertion * OodleCore_Plugins_SetAssertion( t_fp_OodleCore_Plugin_DisplayAssertion * fp_rrDisplayAssertion );
Discussion
Install the callback used by Oodle Core for asserts
Parameters
fp_rrDisplayAssertion  function pointer to your assert display function
Return Value
return  returns the previous function pointer
Discussion

Use this function to install your own display for Oodle Core assertions. This will only happen in debug builds.

The default implementation in debug builds, if you install nothing, uses the C stderr printf for logging, except on Microsoft platforms where it uses OutputDebugString.

WARNING : this function is NOT thread safe! It should be done only once and done in a place where the caller can guarantee thread safety.


 

OodleCore_Plugins_SetPrintfCore pluginst_fp_OodleCore_Plugin_MallocAligned

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignDownS64
Navigation
 OodleX_IOAlignDownS32
 OodleX_IOAlignDownSINTa
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S64 OodleX_IOAlignDownS64( const OO_S64 x );
Discussion
Align down to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x down to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_IOAlignDownS32OodleX UtilsOodleX_IOAlignDownSINTa

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressionLevel
Navigation
 OodleDecompressCallbackRet
 OodleLZ_Jobify
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_CompressionLevel
{
    OodleLZ_CompressionLevel_None = 0,
    OodleLZ_CompressionLevel_SuperFast = 1,
    OodleLZ_CompressionLevel_VeryFast = 2,
    OodleLZ_CompressionLevel_Fast = 3,
    OodleLZ_CompressionLevel_Normal = 4,
    OodleLZ_CompressionLevel_Optimal1 = 5,
    OodleLZ_CompressionLevel_Optimal2 = 6,
    OodleLZ_CompressionLevel_Optimal3 = 7,
    OodleLZ_CompressionLevel_Optimal4 = 8,
    OodleLZ_CompressionLevel_Optimal5 = 9,
    OodleLZ_CompressionLevel_HyperFast1 = -1,
    OodleLZ_CompressionLevel_HyperFast2 = -2,
    OodleLZ_CompressionLevel_HyperFast3 = -3,
    OodleLZ_CompressionLevel_HyperFast4 = -4,
    OodleLZ_CompressionLevel_HyperFast = OodleLZ_CompressionLevel_HyperFast1,
    OodleLZ_CompressionLevel_Optimal = OodleLZ_CompressionLevel_Optimal2,
    OodleLZ_CompressionLevel_Max = OodleLZ_CompressionLevel_Optimal5,
    OodleLZ_CompressionLevel_Min = OodleLZ_CompressionLevel_HyperFast4,
    OodleLZ_CompressionLevel_Force32 = 0x40000000,
    OodleLZ_CompressionLevel_Invalid = OodleLZ_CompressionLevel_Force32
};
Discussion
Selection of compression encoder complexity
Enumerants
OodleLZ_CompressionLevel_None  don't compress, just copy raw bytes
OodleLZ_CompressionLevel_SuperFast  super fast mode, lower compression ratio
OodleLZ_CompressionLevel_VeryFast  fastest LZ mode with still decent compression ratio
OodleLZ_CompressionLevel_Fast  fast - good for daily use
OodleLZ_CompressionLevel_Normal  standard medium speed LZ mode
OodleLZ_CompressionLevel_Optimal1  optimal parse level 1 (faster optimal encoder)
OodleLZ_CompressionLevel_Optimal2  optimal parse level 2 (recommended baseline optimal encoder)
OodleLZ_CompressionLevel_Optimal3  optimal parse level 3 (slower optimal encoder)
OodleLZ_CompressionLevel_Optimal4  optimal parse level 4 (very slow optimal encoder)
OodleLZ_CompressionLevel_Optimal5  optimal parse level 5 (don't care about encode speed, maximum compression)
OodleLZ_CompressionLevel_HyperFast1  faster than SuperFast, less compression
OodleLZ_CompressionLevel_HyperFast2  faster than HyperFast1, less compression
OodleLZ_CompressionLevel_HyperFast3  faster than HyperFast2, less compression
OodleLZ_CompressionLevel_HyperFast4  fastest, less compression
OodleLZ_CompressionLevel_HyperFast  alias hyperfast base level
OodleLZ_CompressionLevel_Optimal  alias optimal standard level
OodleLZ_CompressionLevel_Max  maximum compression level
OodleLZ_CompressionLevel_Min  fastest compression level
OodleLZ_CompressionLevel_Force32 
OodleLZ_CompressionLevel_Invalid 
Discussion
Higher numerical value of CompressionLevel = slower compression, but smaller compressed data.

The compressed stream is always decodable with the same decompressors. CompressionLevel controls the amount of work the encoder does to find the best compressed bit stream. CompressionLevel does not primary affect decode speed, it trades off encode speed for compressed bit stream quality.

I recommend starting with OodleLZ_CompressionLevel_Normal, then try up or down if you want faster encoding or smaller output files.

The Optimal levels are good for distribution when you compress rarely and decompress often; they provide very high compression ratios but are slow to encode. Optimal2 is the recommended level to start with of the optimal levels. Optimal4 and 5 are not recommended for common use, they are very slow and provide the maximum compression ratio, but the gain over Optimal3 is usually small.

The HyperFast levels have negative numeric CompressionLevel values. They are faster than SuperFast for when you're encoder CPU time constrained or want something closer to symmetric compression vs. decompression time. The HyperFast levels are currently only available in Kraken, Mermaid & Selkie. Higher levels of HyperFast are faster to encode, eg. HyperFast4 is the fastest.

The CompressionLevel does not affect decode speed much. Higher compression level does not mean slower to decode. To trade off decode speed vs ratio, use spaceSpeedTradeoffBytes in OodleLZ_CompressOptions


 

OodleDecompressCallbackRetOodleAPI_LZ_CompressorsOodleLZ_Jobify

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Decompress_ThreadPhased_Narrow_Async
Navigation
 OodleXLZ_Decompress_Narrow_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_Decompress_ThreadPhased_Narrow_Async( const void * compBuf,
                                                            OO_SINTa compSize,
                                                            void * decBuf,
                                                            OO_SINTa rawSize,
                                                            OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                                                            void * decBufBase OODEFAULT( NULL ),
                                                            OO_SINTa decBufSize OODEFAULT( 0 ),
                                                            OO_S32 circularBufferBlockCount OODEFAULT( - 1 ),
                                                            void * scratchBuf OODEFAULT( NULL ),
                                                            OO_BOOL synchronous_use_current_thread OODEFAULT( false ) );
Discussion
Start an async LZ decompress for ThreadPhase decoding, using 2 threads
Parameters
compBuf  pointer to compressed data
compBufferSize  number of compressed bytes to decode
rawBuf  pointer to output uncompressed data into
rawLen  number of uncompressed bytes to output
checkCRC  (optional) if data could be corrupted and you want to know about it, pass OodleLZ_CheckCRC_Yes
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder. The decBufBase must be a reset point.
decBufSize  (optional) size of circular buffer starting at decBufBase
circularBufferBlockCount  (optional) number of blocks for circular buffer; generally more is faster but takes more memory; < 0 means use default
scratchBuf  (optional) memory to use for scratch; must be OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded() * circularBufferBlockCount ; if NULL will be allocated
synchronous_use_current_thread  (optional) if true, runs on the current thread and uses 1 additional thread; this makes this a synchronous call and won't return until decompression is done (default is to use 2 worker threads and be fully async)
Return Value
return  OodleXHandle to the operation, wait and check status to get result
Discussion

Runs a 2-thread Narrow decompress using the Oodle Worker system. You must wait and delete the return handle, for example with OodleX_WaitAndDelete

This only works on data that has been compressed with a compressor that's eligible for ThreadPhased decode; check OodleLZ_Compressor_CanDecodeThreadPhased. (currently just Kraken).

See About OodleLZ ThreadPhased Decode

This function (OodleXLZ_Decompress_ThreadPhased_Narrow_Async) does NOT parallelize at seek reset points. You can however do so yourself externally to calling this function. Simply scan the compressed buffer for seek points and launch a separate OodleXLZ_Decompress_ThreadPhased_Narrow_Async call on each seek chunk.

ThreadPhased decode is always fuzz safe.

If synchronous_use_current_thread then the returned handle is not async, you may check its status to get the result.


 

OodleDecompressCallback_WriteFile_DataOodleXAPI_LZ_AsyncOodleXLZ_Decompress_Narrow_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
Navigation
 OODLE_WORKERS_COUNT_ALL_HYPER_CORES
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES (-1)
Discussion
Make workers for every physical core eg. in a 6-physical core, 12-hyper-thread system, would make 6 threads this is usually best for Oodle Data LZ compression work See also OODLE_WORKERS_COUNT_ALL_HYPER_CORES
 
OodleX Startup and ShutdownOodleX Startup and ShutdownOODLE_WORKERS_COUNT_ALL_HYPER_CORES

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_MustDecodeWithoutResets
Navigation
 OodleLZ_Compressor_CanDecodeInPlace
 OodleLZ_Compressor_CanDecodeFuzzSafe
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_MustDecodeWithoutResets( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor must decode contiguous ranges of buffer with the same Decoder.

That is, most of the compressors can be Reset and restart on any block, not just seek blocks, as long as the correct window data is provided. That is, if this returns false then the only state required across a non-reset block is the dictionary of previously decoded data.

But if OodleLZ_Compressor_MustDecodeWithoutResets returns true, then you cannot do that, because the Decoder object must carry state across blocks (except reset blocks).

This does not apply to seek points - you can always reset and restart decompression at a seek point.
 

OodleLZ_Compressor_CanDecodeInPlaceOodleAPI_LZ_CompressorsOodleLZ_Compressor_CanDecodeFuzzSafe

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State
Navigation
 OodleNetwork1TCP_State
 OodleNetwork1UDP_StateCompacted
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleNetwork1UDP_State;
Discussion
Opaque type for an OodleNetwork1UDP_State
Discussion
OodleNetwork1UDP uses a OodleNetwork1_Shared just like the non-UDP OodleNetwork1
 
OodleNetwork1TCP_StateOodleAPI_OodleNetwork1OodleNetwork1UDP_StateCompacted

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_WaitAndDestroyThread
Navigation
 OodleX_CreateThread
 OodleX_ReleaseThreadTLS
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_WaitAndDestroyThread( OodleX_Thread t );
Discussion
Wait on thread being complete and free all resources
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX_CreateThreadOodleX threading utilOodleX_ReleaseThreadTLS

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Decode_ThreadPhase
Navigation
 OodleLZ_Jobify
 OodleLZ_FuzzSafe
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_Decode_ThreadPhase
{
    OodleLZ_Decode_ThreadPhase1 = 1,
    OodleLZ_Decode_ThreadPhase2 = 2,
    OodleLZ_Decode_ThreadPhaseAll = 3,
    OodleLZ_Decode_Unthreaded = OodleLZ_Decode_ThreadPhaseAll
};
Discussion
ThreadPhase for threaded Oodle decode
Discussion
Check OodleLZ_Compressor_CanDecodeThreadPhased (currently only used by Kraken)

See About OodleLZ ThreadPhased Decode


 

OodleLZ_JobifyOodleAPI_LZ_CompressorsOodleLZ_FuzzSafe

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_Shared_SetWindow
Navigation
 OodleNetwork1_CompressedBufferSizeNeeded
 OodleNetwork1TCP_State_Reset
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNetwork1_Shared_SetWindow( OodleNetwork1_Shared * data,
                                     OO_S32 htbits,
                                     const void * window,
                                     OO_S32 window_size );
Discussion
Fill a OodleNetwork1_Shared from provided data
Parameters
data  OodleNetwork1_Shared object to fill
htbits  size of the OodleNetwork1 hash table (log2) ; typically 18-21 such as OODLENETWORK1_HASH_BITS_DEFAULT
window  bytes of static dictionary data to use for compression
window_size  size of window ; should be <= OODLENETWORK1_MAX_DICTIONARY_SIZE
Discussion

This must be done on both the client and server to fill the OodleNetwork1_Shared object used for compression.

window should be some typical transmitted data. The better you can make it represent the common data seen, the better compression will be. The most common types of packets should be placed at the end of the window.

You must load window from disk somehow, or it could be data that you already have in memory for some other purpose - any data which both the client and server have exactly the same copy of can be used as the compression dictionary. To save and load the window from disk you should generally use one of the standard About OodleLZ compressors.

NOTE : window is not copied ; do not free it! OodleNetwork1_Shared keeps pointers into window, it must be kept allocated while this OodleNetwork1_Shared is in use.

You may call SetWindow multiple times on the same Shared data for purposes of training.


 

OodleNetwork1_CompressedBufferSizeNeededOodleAPI_OodleNetwork1OodleNetwork1TCP_State_Reset

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_CanDecodeFuzzSafe
Navigation
 OodleLZ_Compressor_MustDecodeWithoutResets
 OodleLZ_Compressor_RespectsDictionarySize
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_CanDecodeFuzzSafe( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor is "fuzz safe" which means it can accept corrupted data and won't crash or overrun any buffers.
 
OodleLZ_Compressor_MustDecodeWithoutResetsOodleAPI_LZ_CompressorsOodleLZ_Compressor_RespectsDictionarySize

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Android
Navigation
 About Oodle on IOS
 About Oodle on Platforms
 About Oodle on WASM
 Welcome to Oodle
 Change Log

Oodle on Android is currently provided as only the Oodle Core lib (no OodleX).

This includes the synchronous LZ compressors, as well as Oodle Network compression.

See Oodle2 Core vs Oodle2 Ext


Oodle on mobile does not include the Optimal level encoders. When OodleLZ_CompressionLevel_Optimal1 or higher is requested, OodleLZ_CompressionLevel_Normal is used instead.
 

About Oodle on IOSAbout Oodle on PlatformsAbout Oodle on WASM

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle
Navigation
 Welcome to Oodle
 Change Log

About Oodle

Oodle is a family of solutions for efficient data compression.

Oodle consists of :

Oodle Data : generic lossless data compression (Kraken, Leviathan, Mermaid, and Selkie)

Oodle Network : packet compression to reduce bandwidth (see About Oodle Network Compression or Getting Started with Oodle Network).

Oodle Texture : encoding for BCN block-compressed GPU textures that dramatically reduces their size.

Oodle Lossless Image : specialized encoder for lossless RGB image encoding. Faster and smaller than PNG.

Oodle Data and Network are both described in this help, but are distributed as separate SDKs. Oodle Texture and Oodle Lossless Image are distributed separately.

Oodle Data SDK comes as two libraries - Core and Ext. Core provides synchronous memory->memory compressors that require no initialization and are very portable. Ext provides helpers for async compression and file IO. Most games should use Oodle2 Core in their runtime and Oodle2 Ext in their tool chain. (see Oodle2 Core vs Oodle2 Ext)

Oodle Data compresses packages for distribution with state of the art lossless (LZ) data compression. Oodle provides many different levels of CPU use to compression ratio tradeoffs. Oodle lets you use the same data compression and packaging formats on all platforms. (see About OodleLZ or Getting Started with Oodle LZ Data Compression)

Further Reading

See also :

Frequently Asked Questions

Index of Abouts

Introducing the new Oodle Leviathan

Getting Started with Oodle Network

Getting Started with Oodle LZ Data Compression

Oodle2 Core vs Oodle2 Ext

Oodle Network and Data SDK separation

  •  Index of Abouts
  •  About Oodle ozip
  •  About Oodle on Platforms
    •  About Oodle on Windows
      •  About Oodle on Windows UWP
    •  About Oodle on PS4
    •  About Oodle on Nintendo Switch
    •  About Oodle on Mac
    •  About Oodle on Xbox One
    •  About Oodle on Linux
    •  About Oodle on IOS
    •  About Oodle on Android
    •  About Oodle on WASM
  •  About Oodle Job Threading Plugins
  •  About Compression Scratch Memory

 
  Index of Abouts

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetOSFileOps
Navigation
 OodleXIOQ_NameIsDir_AsyncAndWait
 OodleX_GetDefaultFileOps
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleXFileOpsVTable * OodleX_GetOSFileOps( );
Discussion
Return a const OodleXFileOpsVTable with the base OS implementations
Discussion
Contains the base file ops functions for the current OS.

Do not change this struct!
 

OodleXIOQ_NameIsDir_AsyncAndWaitOodleX low level async ioOodleX_GetDefaultFileOps

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_SetInfoByName_Async
Navigation
 OodleXIOQ_GetInfoByName_GetResult
 OodleXIOQ_ReadMallocWholeFile_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_SetInfoByName_Async( const char * name,
                                            OO_U32 flags,
                                            OO_U64 modTime,
                                            OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                            OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                            const OodleXHandle * dependencies OODEFAULT( NULL ),
                                            OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an asynchronous SetInfo request
Parameters
name  the file name to query (VFS,UTF-8)
flags  file flags (logical OR of OODLEX_FILEINFO_FLAGS) , or OODLEX_FILEINFO_FLAG_INVALID to leave unchanged
modTime  mod time to change, or %OODLEX_FILEINFO_MODTIME_INVALID to leave uncahnged
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Used to set flags or modtime on a file. Flags and modTime correspond to OodleXFileInfo:flags and OodleXFileInfo:modTime.

All members of OodleXFileInfo can be set this way, except size; to set size use OodleXIOQ_SetFileSize_Async.
 

OodleXIOQ_GetInfoByName_GetResultOodleX low level async ioOodleXIOQ_ReadMallocWholeFile_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_OodleFPVoidVoid
Navigation
 t_OodleFPVoidVoidStar
 Welcome to Oodle
 Change Log
// Function typedef:
void( OODLE_CALLBACK t_OodleFPVoidVoid )( void );
Discussion
void-void callback func pointer takes void, returns void
 
Oodle_LogHeaderCore Baset_OodleFPVoidVoidStar

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CheckCRC
Navigation
 OodleLZ_PackedRawOverlap
 OodleLZ_Profile
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_CheckCRC
{
    OodleLZ_CheckCRC_No = 0,
    OodleLZ_CheckCRC_Yes = 1,
    OodleLZ_CheckCRC_Force32 = 0x40000000
};
Discussion
Bool enum for the LZ decoder - should it check CRC before decoding or not?
Discussion
NOTE : the CRC's in the LZH decompress checks are the CRC's of the compressed bytes. This allows checking the CRc prior to decompression, so corrupted data cannot be fed to the compressor.

To use OodleLZ_CheckCRC_Yes, the compressed data must have been made with OodleLZ_CompressOptions:sendQuantumCRCs set to true.

If you want a CRC of the raw bytes, there is one optionally stored in the OodleLZ_SeekTable and can be confirmed with OodleLZ_CheckSeekTableCRCs
 

OodleLZ_PackedRawOverlapOodleAPI_LZ_CompressorsOodleLZ_Profile

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFileOpenFlags
Navigation
 OodleXFileMode
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXFileOpenFlags
{
    OodleXFileOpenFlags_Default = 0,
    OodleXFileOpenFlags_Buffered = 1,
    OodleXFileOpenFlags_NotBuffered = 2,
    OodleXFileOpenFlags_WriteCreateDontStomp = 4,
    OodleXFileOpenFlags_Force32 = 0x40000000
};
Discussion
OodleXFileOpenFlags specify options when opening files
Enumerants
OodleXFileOpenFlags_Default  use Oodle's default for this platform
OodleXFileOpenFlags_Buffered  use an OS-buffered file
OodleXFileOpenFlags_NotBuffered  use a non-OS-buffered file, when possible
OodleXFileOpenFlags_WriteCreateDontStomp  Open for WriteCreate by default stomps existing; this prevents it
OodleXFileOpenFlags_Force32 
Discussion
OodleXFileOpenFlags_Default lets Oodle select buffered or unbuffered based on the system and global settings.

OodleXFileOpenFlags_Buffered files are guaranteed to work with unaligned IO.

OodleXFileOpenFlags_Default and OodleXFileOpenFlags_NotBuffered files require aligned IO on some platforms.

Flags may be combined with logical OR
 

OodleXFileModeOodleX low level async ioOodleXFileOpsVTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Compress_AsyncAndWait
Navigation
 OodleXLZ_Compress_Wait_GetResult
 OodleXLZ_ReadAndDecompress_Stream_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleXLZ_Compress_AsyncAndWait( OO_U32 asyncSelect,
                                         OodleLZ_Compressor compressor,
                                         const void * rawBuf,
                                         OO_SINTa rawLen,
                                         void * compBuf,
                                         OodleLZ_CompressionLevel compressSelect,
                                         const OodleLZ_CompressOptions * pOptions OODEFAULT( NULL ),
                                         const void * dictionaryBase OODEFAULT( NULL ) );
Discussion
Does OodleXLZ_Compress_Async and OodleXLZ_Compress_Wait_GetResult
 
OodleXLZ_Compress_Wait_GetResultOodleXAPI_LZ_AsyncOodleXLZ_ReadAndDecompress_Stream_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Decompress_MakeSeekTable_Wide_Async
Navigation
 OodleXLZ_Decompress_Wide_Async
 OodleXLZ_Compress_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXLZ_Decompress_MakeSeekTable_Wide_Async( OO_U32 asyncSelect,
                                                           OO_S32 seekChunkLen,
                                                           const void * packedDataPtr,
                                                           OO_SINTa packedLen,
                                                           void * rawArray,
                                                           OO_SINTa rawArrayLen,
                                                           OodleLZ_FuzzSafe fuzzSafe OODEFAULT( OodleLZ_FuzzSafe_No ),
                                                           OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                                                           OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                                                           void * decBufBase OODEFAULT( NULL ),
                                                           OO_SINTa decBufSize OODEFAULT( 0 ),
                                                           OodleLZ_PackedRawOverlap packedRawOverlap OODEFAULT( OodleLZ_PackedRawOverlap_No ),
                                                           OodleXIOQFile writeToFile OODEFAULT( 0 ),
                                                           OO_S64 writeToFileStartPos OODEFAULT( 0 ),
                                                           OodleXHandle * pWriteHandleGroup OODEFAULT( 0 ),
                                                           OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                           const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                           OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start an async LZ decompress, possibly write raw data
Parameters
asyncSelect  logical OR of OodleXAsyncSelect flags determine how the async is run
seekChunkLen  length of seekChunks used in compression OodleLZ_CompressOptions:seekChunkLen
packedDataPtr  pointer to LZ compressed data
packedLen  compressed data length
rawArray  pointer to memory filled with decompressed data
rawArrayLen  length of decompressed data
checkCRC  if OodleLZ_CheckCRC_Yes, the decompressor checks the crc to ensure data integrity
verbosity  (optional) if not OodleLZ_Verbosity_None, will log some information
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder.
decBufSize  (optional) size of circular buffer starting at decBufBase
packedRawOverlap  (optional) if OodleLZ_PackedRawOverlap_Yes, the compressed data is in the same memory array as the output raw data
writeToFile  (optional) OodleXIOQFile to write raw data to
writeToFileStartPos  (optional) file position where writeToFile should start (must be OODLEX_IO_MAX_ALIGNMENT aligned)
pWriteHandleGroup  (optional) if writeToFile is given, this is filled with an OodleAsyncGroup OodleXHandle containing all the file IO operations
autoDelete  (optional) see OodleXHandleAutoDelete
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  OodleXHandle to the operation, or OodleXHandle_Null for invalid arguments
Discussion

Same as OodleXLZ_Decompress_Wide_Async , but makes the seek table for you.

Can be used as a drop-in replacement for OodleLZ_Decompress() but with parallel decoding.

If the data is not parallel-decodable (because it has no seek resets, eg. OodleLZ_CompressOptions:seekChunkReset was not set) this is slower than just calling OodleLZ_Decompress. So this should only be used when you believe parallel decoding is possible.

seekChunkLen most follow the rules for Oodle seek chunk lengths. See OodleLZ_MakeSeekChunkLen. It should be a power of two and greater-equal than OODLELZ_BLOCK_LEN.
 

OodleXLZ_Decompress_Wide_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Compress_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compress
Navigation
 OodleLZ_Decompress
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_Compress( OodleLZ_Compressor compressor,
                           const void * rawBuf,
                           OO_SINTa rawLen,
                           void * compBuf,
                           OodleLZ_CompressionLevel level,
                           const OodleLZ_CompressOptions * pOptions OODEFAULT( NULL ),
                           const void * dictionaryBase OODEFAULT( NULL ),
                           const void * lrm OODEFAULT( NULL ),
                           void * scratchMem OODEFAULT( NULL ),
                           OO_SINTa scratchSize OODEFAULT( 0 ) );
Discussion
Compress some data from memory to memory, synchronously, with OodleLZ
Parameters
compressor  which OodleLZ variant to use in compression
rawBuf  raw data to compress
rawLen  number of bytes in rawBuf to compress
compBuf  pointer to write compressed data to. MUST be at least OodleLZ_GetCompressedBufferSizeNeeded bytes
level  OodleLZ_CompressionLevel controls how much CPU effort is put into maximizing compression
pOptions  (optional) options; if NULL, OodleLZ_CompressOptions_GetDefault is used
dictionaryBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder.
lrm  (optional) long range matcher
scratchMem  (optional) pointer to scratch memory
scratchSize  (optional) size of scratch memory (see OodleLZ_GetCompressScratchMemBound)
Return Value
return  size of compressed data written, or OODLELZ_FAILED for failure
Discussion

Performs synchronous memory to memory LZ compression.

In tools and when compressing large inputs in one call, consider using OodleXLZ_Compress_AsyncAndWait (in the Oodle2 Ext lib) instead to get parallelism. Alternatively, chop the data into small fixed size chunks (we recommend at least 256KiB, i.e. 262144 bytes) and call compress on each of them, which decreases compression ratio but makes for trivial parallel compression and decompression.

You can compress a large buffer in several calls by setting dictionaryBase to the start of the buffer, and then making rawBuf and rawLen select portions of that buffer. As long as rawLen is a multiple of OODLELZ_BLOCK_LEN, the compressed chunks can simply be concatenated together.

The buffer that compBuf points to must have a certain minimum size that is returned by OodleLZ_GetCompressedBufferSizeNeeded. This size is always more than rawLen, usually by a few bytes for every 256kb of input data. The "compressed" data can end up slightly larger than the input due to internal stream headers.

If scratchMem is provided, it will be used for the compressor's scratch memory needs before OodleMalloc is called. If the scratch is big enough, no malloc will be done. If the scratch is not big enough, the compress will not fail, instead OodleMalloc will be used. OodleMalloc should not return null. There is currently no way to make compress fail cleanly due to using too much memory, it must either succeed or abort the process.

If scratchSize is at least OodleLZ_GetCompressScratchMemBound , additional allocations will not be needed.

See About OodleLZ for tips on setting the compression options.

If dictionaryBase is provided, the backup distance from rawBuf must be a multiple of OODLELZ_BLOCK_LEN

If OodleLZ_CompressOptions:seekChunkReset is enabled, and dictionaryBase is not NULL or rawBuf , then the seek chunk boundaries are relative to dictionaryBase, not to rawBuf.


 

OodleLZ_SeekTableOodleAPI_LZ_CompressorsOodleLZ_Decompress

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_State_Size
Navigation
 OodleNetwork1_Shared_Size
 OodleNetwork1_CompressedBufferSizeNeeded
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1TCP_State_Size( void );
Discussion
Returns the size of memory required for an OodleNetwork1TCP_State object
Discussion
Shared and State are allocated with malloc( Size() )


 

OodleNetwork1_Shared_SizeOodleAPI_OodleNetwork1OodleNetwork1_CompressedBufferSizeNeeded

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignDownSINTa
Navigation
 OodleX_IOAlignDownS64
 OodleX_S64_to_SINTa_check
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleX_IOAlignDownSINTa( const OO_SINTa x );
Discussion
Align down to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x down to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_IOAlignDownS64OodleX UtilsOodleX_S64_to_SINTa_check

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_CopyFile_AsyncAndWait
Navigation
 OodleXIOQ_WriteWholeFile_AsyncAndWait
 OodleXIOQ_GetInfoByName_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_CopyFile_AsyncAndWait( const char * from,
                                         const char * to,
                                         OO_U32 oodleCopyFileFlags );
Discussion
See OodleXIOQ_CopyFile_Async
 
OodleXIOQ_WriteWholeFile_AsyncAndWaitOodleX low level async ioOodleXIOQ_GetInfoByName_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressOptions_Validate
Navigation
 OodleLZ_CompressOptions_GetDefault
 OodleLZ_Compressor_UsesWholeBlockQuantum
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleLZ_CompressOptions_Validate( OodleLZ_CompressOptions * pOptions );
Discussion
Clamps the values in pOptions to be in valid range
 
OodleLZ_CompressOptions_GetDefaultOodleAPI_LZ_CompressorsOodleLZ_Compressor_UsesWholeBlockQuantum

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Network plugins
Navigation
 Oodle Network compression
 Oodle2 Network API Documentation
 Welcome to Oodle
 Change Log
  •  Functions
    •  OodleNet_Plugins_SetAllocators
    •  OodleNet_Plugins_SetJobSystem
    •  OodleNet_Plugins_SetJobSystemAndCount
    •  OodleNet_Plugins_SetPrintf
    •  OodleNet_Plugins_SetAssertion
  •  Typedefs
    •  t_fp_OodleNet_Plugin_MallocAligned
    •  t_fp_OodleNet_Plugin_Free
    •  t_fp_OodleNet_Plugin_RunJob
    •  t_fp_OodleNet_Plugin_WaitJob
    •  t_fp_OodleNet_Plugin_Printf
    •  t_fp_OodleNet_Plugin_DisplayAssertion

 
About Oodle Network CompressionOodle2 Network API DocumentationOodleNet_Plugins_SetAllocators

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX Memory Allocators
Navigation
 OodleX Startup and Shutdown
 Oodle2 Ext API Documentation
 OodleX async handle operations
 Welcome to Oodle
 Change Log

OodleXMalloc memory allocators.

  •  About OodleXMalloc
  •  Enumerants
    •  OodleXMalloc_OS_Options
  •  Structures
    •  OodleXMallocVTable
  •  Functions
    •  OodleXMalloc_InstallVTable
    •  OodleXMalloc_SetFailedHandler
    •  OodleXMalloc
    •  OodleXMallocAligned
    •  OodleXFree
    •  OodleXFreeSized
    •  OodleXMallocBigAlignment
    •  OodleXMallocBig
    •  OodleXFreeBig
    •  OodleXMalloc_ValidatePointer
    •  OodleXMalloc_IOAligned
    •  OodleXFree_IOAligned
    •  OodleXMalloc_GetVTable_Clib
    •  OodleXMalloc_GetVTable_OS
  •  Typedefs
    •  OodleXMallocFailedHandler

 
OodleXConfigValuesOodle2 Ext API DocumentationAbout OodleXMalloc

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleLZ Hydra
Navigation
 About OodleLZ ThreadPhased Decode
 Core LZ compression
 OodleAPI_LZ_Compressors
 Welcome to Oodle
 Change Log

About OodleLZ Hydra

Oodle Hydra is a meta-compressor which automatically selects Leviathan, Kraken, Mermaid, or Selkie on a per-block basis.

When you decode a file that was compressed with Hydra, it will decode as one or several of those other compressors.

Hydra makes its decision by scoring each compressor for its space-speed performance. What this means is Hydra automatically makes good decisions about using the slower compressors only when they are worth it. That is, they must provide a good return in terms of bytes saved in exchange for the increase in decode time. Exactly what qualifies as "worth it" is determined by you, via the OodleLZ_CompressOptions:spaceSpeedTradeoffBytes parameter.

With spaceSpeedTradeoffBytes around 256 , Hydra is roughly comparable to Kraken. As you dial it lower, Hydra will give more compression but slower decodes. Around 1500 is comparable to Mermaid, and around 50 is comparable to Leviathan. In between you can hit a balance that's somewhere between those compressors.

Hydra takes more time to encode because it is considering many compressors.

Hydra always beats every other Oodle compressor, it provides the best of them.
 

About OodleLZ ThreadPhased DecodeCore LZ compressionOodleAPI_LZ_Compressors

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How does OodleLZ compare with other compressors ?
Navigation
 FAQ: Do new Oodle versions break data compatibility ?
 Frequently Asked Questions
 FAQ: Which OodleLZ should I use?
 Welcome to Oodle
 Change Log

In general, Oodle LZ is designed to get compression as good as anything in the world, while decoding much MUCH faster. Usually 2-10X faster!

Kraken is the compressor you should try first. It offers excellent compression ratio and very high decode speeds. It's a great compromise for game data loading.

Leviathan gets a bit more compression than Kraken (similar to 7zip/LZMA, or Oodle LZNA), but is a bit slower to decode than Kraken. It's way faster than anything else with similar compression. Leviathan can be a bit slow to encode.

Mermaid & Selkie are some of the fastest decompressors in the world. They provide less compression than Kraken but super fast decodes.

Mermaid & Selkie (at the fast OodleLZ_CompressionLevel settings) are also very fast to encode.

Here's a chart showing the compression ratio of various files of different types :

And a chart of the decode speed :

(all compressors were run in max compress mode; per-file hand tweakable of options was not done for any of the compressors; multimedia and x86 filters were disabled for all compressors)

The best way for you to evaluate the Oodle compressors is just to run them on your own data and see what you get. Try example_lz_chart : Example that makes a chart of OodleLZ options (in the bin dir) for an easy way to get a report of Oodle's performance.
 

FAQ: Do new Oodle versions break data compatibility ?Frequently Asked QuestionsFAQ: Which OodleLZ should I use?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetCompressScratchMemBoundEx
Navigation
 OodleLZ_GetCompressScratchMemBound
 OodleLZ_GetCompressedBufferSizeNeeded
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetCompressScratchMemBoundEx( OodleLZ_Compressor compressor,
                                               OodleLZ_CompressionLevel level,
                                               OodleLZ_CompressScratchMemBoundType boundType,
                                               OO_SINTa rawLen,
                                               const OodleLZ_CompressOptions * pOptions OODEFAULT( NULL ) );
Discussion
Return an estimate for the amount of scratch mem used by OodleLZ_Compress
Parameters
compressor  which OodleLZ variant to use in compression
level  OodleLZ_CompressionLevel controls how much CPU effort is put into maximizing compression
boundType  Type of memory estimate to return.
rawLen  maximum number of bytes you will compress (plus dictionary backup)
pOptions  (optional) options; if NULL, OodleLZ_CompressOptions_GetDefault is used
Discussion

Returns either a worst-case or typical scratch memory estimate for the given compressor, options and input size.

When a worst-case scratch memory estimate exists, passing that much scratch memory to OodleLZ_Compress is guaranteed to not do any allocations, provided the parameters match those of the compression call.

"Typical" memory bounds are not hard guarantees but will usually not result in any extra allocations.

For rawLen pass at least the maximum size you will ever encode. If your data is divided into chunks, pass the chunk size. If you will encode full buffers of unbounded size, pass -1.

Some options and levels may not have simple finite bounds. Then OODLELZ_SCRATCH_MEM_NO_BOUND is returned and the call to OodleLZ_Compress may use the allocator even if infinite scratch memory is provided. Currently this applies to all the Optimal levels.

When OODLELZ_SCRATCH_MEM_NO_BOUND is returned, you can still pass in scratch mem which will be used before going to the plugin allocator.


 

OodleLZ_GetCompressScratchMemBoundOodleAPI_LZ_CompressorsOodleLZ_GetCompressedBufferSizeNeeded

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX Startup and Shutdown
Navigation
 OodleX LZ compression
 Oodle2 Ext API Documentation
 OodleX Memory Allocators
 Welcome to Oodle
 Change Log
  •  Defines
    •  OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
    •  OODLE_WORKERS_COUNT_ALL_HYPER_CORES
  •  Enumerants
    •  OodleX_Init_GetDefaults_DebugSystems
    •  OodleX_Init_GetDefaults_Threads
    •  OodleX_Shutdown_LogLeaks
    •  OodleX_Shutdown_DebugBreakOnLeaks
  •  Structures
    •  OodleXInitOptions
    •  OodleXConfigValues
  •  Functions
    •  OodleX_Init_ThreadProfilerInit
    •  OodleX_Init_GetDefaults
    •  OodleX_Init_GetDefaults_Minimal
    •  OodleX_Init
    •  OodleX_Init_Default
    •  OodleX_LogSystemInfo
    •  OodleX_Shutdown
    •  OodleX_Init_NoThreads
    •  OodleX_Shutdown_NoThreads
    •  OodleX_GetConfigValues
    •  OodleX_SetConfigValues

 
OodleXAPI_LZ_AsyncOodle2 Ext API DocumentationOODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Tips for benchmarking a compressor
Navigation
 Welcome to Oodle
 Change Log

You're about to evaluate Oodle (thanks for having a look!) or some other compressor. Before you start, consider these tips :

  • Time only the compressor.

Place your time measurements only around the compressor. Not IO, not your parsing, not mallocs, just the compress or decompress calls. I understand that in the end what you care about is total time to load, but there can be a lot of issues there that need fixing, and they can cloud the comparison of just the compression part. eg. if your parsing is really slow, that will dominate the CPU time and hide the differences between the compressors.

  • Time what you actually care about.

If you care about decode time, time the decompression. If you care about encode time, time compression. If you care about round-trip time, add the two times. Compressors are not just "fast" or "slow" at both ends, you can't time encoding and decide that it's a fast or slow compressor if what you care about is decoding.

  • Choose the right options.

Most compressors have the ability to target slightly different use cases. The most common option is the ability to trade off encode time vs. compression ratio. So, if what you care about is smallest size, then run the compressor at its highest encode effort level. It can be tricky to get the options right in most compression libraries; we are woefully non-standardized and not well documented. Aside from the simple "level" parameter, there may be other options that are relevant to your goals, perhaps trading off decompressor memory usage, or decompression speed. With Oodle the best option is always to email us and ask what options will best suit your goals.

  • Run apples-to-apples (threads-to-threads) comparisons.

It can be tricky to compare compressors fairly. As much as possible they should be run in the same way, and they should be run in the way that you will actually use them in your final application. Don't profile them with threads if you will not use them threaded in your shipping application.

Threads are a common problem. Compressors should either be tested all threaded (if you will use threads in your final application), or all non-threaded. Unfortunately the defaults are not the same. "lzma" (7z) and LZHAM create threads by default. You have to change their options to tell them to not create threads. The normal Oodle_Compress calls will not use threads by default, you have to specifically call one of the Async threaded routines. So either set everything to not use threads, or set everything to use threads, so the comparison is on equal footing. (my personal preference is to benchmark everything without threads to compare single-threaded performance, and you can always add threads for production use)

  • Take the MIN of N run times.

To get reliable timing, you need to run the loop many times, and take the MIN of all times. The min will give you the time it takes when the OS isn't interrupting you with task switches, the CPU isn't clocking-down for speedstep, etc. I usually do 30 per core but you can probably get a way with a bit less.

On some modern cores that do short term boost, or on ARM big/little cores, it can be better to take the MEDIAN (not the average) of many runs, or to exclude the top 10% fastest runs then take the min.

If possible, use BIOS and OS settings to put the procesor in a more stable mode.

  • Wipe the cache.

Assuming you are now doing N loops, you need to invalidate the cache between iterations. If you don't, you will be running the compressor in a "hot cache" scenario, with some buffers already in cache. Most likely the real world performance you actually care about is the cold cache performance, so invalidate the cache in the timing loop.

This has the biggest effect on small buffers. It's fine to do your timing in either hot cache or cold cache, the important thing is consistency - don't care numbers from a hot cache run vs a cold cache run, particularly on small buffers.

  • Don't pack a bunch of files together in a tar if that's not how you load.

It may seem like a good way to test to grab your bunch of test files and pack them together in a tar (or zip -0 or similar package) and run the compression tests on that tar. That's a fine option if that's really how you load data in your final application - as one big contiguous chunk that must be loaded in one big blob. But most people don't. You need to test the compressors in the same way they will be used in the final application. If you load whole file at a time, test the compressors on whole file units. Many people do loading on some kind of paging unit, like perhaps 1 MB chunks. If you do that, then test the compressor on the same thing.

  • Choose your test set to proportionally represent your real data load.

If you could test on the entire set of buffers that your final application will load, that would be an accurate test. (though actually, even that is a bit subtle, since some buffers are more latency sensitive than others, so for example you might care more about the first few things you load to get into a running application as quickly as possible). That's probably not practical, so you want to choose a set that is representative of what you will actually load. Don't exclude things like already-compressed files (JPEGs and so on) if you will be running them through the compressor. (though consider not running them your compressed-file loading path, in which case you should exclude them from testing). It's pretty hard to get an accurate representative sample, so it's generally best to just get a variety of files and look at individual per-file results.

A common mistake is to test on standard corpora like Silesia or Canterbury that have a majority of their data in ASCII text. You probably do not the majority of your bytes spent on text, so those are very poor reflections or real world data loads.

  • Look at the spectrum of results, not the sum.

After you run on your test set, don't just add up the compressed sizes and times to make a "total" result. Sums can be misleading. One issue is there are some large incompressible files, they can hide the differences on the more compressible files. But a bigger and more subtle trap is the way that sums weight the combination of results. A sum is a weighting by the size of each file in the test set. That's fine if your test set is all of your data, or is a perfectly proportionally representative sampling of all of your data (a subset which acts like the whole). But most likely it's not. It's best to keep the results per file separate and just have a look at individual cases to see what's going on, how the results differ, and try not to simplify to just looking at the sum.

  • If you do sum, sum time not speed, sum size not ratio.

Speed (like mb/s) and ratio (raw size/comp size) are inverted measures and shouldn't be summed. What you actually care about is total compressed size, and total time to decode. So if you run over a set of files, don't look at "average speed" or "average ratio" , because those are inverted meaures that will oddly weight the accumulation. Instead accumulate total time to decode, total raw size, and total compressed size, and then if you like you can make "overall speed" and "overall ratio" from those total.

  • Try not to malloc in the timing loop.

Your malloc might be fast, it might be slow, it's best to not have that as a variable in the timing. In general try to allocate the memory for the compressor or decompressor outside of the timing loop. (In Oodle this is done by passing in your own pointer for the "decoderMemory" argument of OodleLZ_Decompress). That would be an unfair test if you didn't also do that in the final application - so do it in the final application too! (similarly, make sure there's no logging inside the timing loop).

  • Consider excluding almost-incompressible files.

This is something you should consider for final shipping application, and if you do it in your shipping application, then you should do it for the benchmark too. The most common case is already-compressed files like JPEG images and MP3 audio. These files can usually be compressed slightly, maybe saving 1% of their size, but the time to decode them is not worth it overall - you can get more total size savings by running a more powerful compressor on other files. So it's most efficient to just send them uncompressed.

  • Tiny files should either be excluded or packed together.

There's almost never a use case where you really want to compress tiny files (< 16k bytes or so) as independent units. There's too much per-unit overhead in the compressor, and more importantly there's too much per-unit overhead in IO - you don't want to eat a disk seek to just to get one tiny file. So in a real application tiny files should always be grouped into paging units that are 256k or more, a size where loading them won't just be a total waste of disk seek time. So, when benchmarking compressors you also shouldn't run them on tiny independent files, because you will never do that in a shipping application.

  • Beware of trying to time IO.

It is attractive to try to time the entire loading process (IO + decompression + processing) to see how Oodle affects your total load time. While this is a nice idea, it is fraught with peril in practice. IO timing is notoriously inconsistent, and the times you measure may not reflect real user times at all. (for example, if you measuring timing repeatedly, you may be just loading data from the system disk cache; on a console you may be timing through a simulator or host FS which doesn't reflect real speeds; etc.) Because of this I recommend timing just the compression portion of the loading operation, or at least timing both aspects. To really time IO right you need to make many runs on different hardware, after fresh reboots, it's quite a lot of work.

  • Compare to the provided Oodle executables

Oodle provides example_lz_chart : Example that makes a chart of OodleLZ options and ozip (About Oodle ozip) as pre-built executables (in the bin dir) that can be used for benchmarking. Test your code to make sure your results agree with them. They are also available as source code so you can see how the benchmarking is done.

  • One good number is better than a million bad ones.

Before you go and try to be all "big data" and run over a giant corpus, make sure you can benchmark just one file and get correct results. Compare it to the published Oodle benchmark results, which are very carefully made, and if it doesn't agree make sure you know why. Oodle is run on some public domain data sets, such as the Silesia corpus, which you can get here :

http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia

if your numbers on Silesia don't agree with the standard Oodle numbers, look into why.
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenWriteWholeFileClose_Async
Navigation
 OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
 OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenWriteWholeFileClose_Async( const char * name,
                                                      const void * buffer,
                                                      OO_SINTa size,
                                                      OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                      const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                      OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                      OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                      const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                      OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to open a file, write a buffer, and close it.
Parameters
name  name of the file to open (VFS, UTF-8)
buffer  the buffer to write (must be OODLEX_IO_MAX_ALIGNMENT aligned)
size  the final file size (no alignment required)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

Performs OodleXIOQ_OpenForWriteCreate_Async, OodleXIOQ_Write_Async, and OodleXIOQ_CloseFile_Async.

You might also want to enqueue a OodleXIOQ_FreeBufferIOAligned_Async after this, but it is not done for you. See also FAQ: I write a file with IOQ but the contents are garbage?.

The OodleXHandle returned is not done until the entire compound operation is done.
 

OodleXIOQ_OpenAndReadMallocWholeFileAndClose_AsyncOodleX low level async ioOodleXIOQ_OpenWriteWholeFileCloseTempName_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX Debug aids
Navigation
 OodleX low level async io
 Oodle2 Ext API Documentation
 OodleX Utils
 Welcome to Oodle
 Change Log

Oodle API's to help with debugging and diagnostics.

  •  Defines
    •  OodleXLog_Printf
  •  Enumerants
    •  OodleXLog_StateFlags
    •  OodleXLog_VerboseLevel
    •  OodleXLogCallbackRetRet
  •  Functions
    •  OodleXLog_SetState
    •  OodleXLog_SetEcho
    •  OodleXLog_GetEcho
    •  OodleXLog_SetCallback
    •  OodleXLog_GetCallback
    •  OodleXLog_GetVerboseLevel
    •  OodleXLog_SetVerboseLevel
    •  OodleXLog_Flush
    •  OodleXLog_PrintfError
  •  Typedefs
    •  OodleXLogCallbackRet

 
OodleX_SetDefaultFileOpsOodle2 Ext API DocumentationOodleXLog_Printf

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
How to build and use the Oodle examples
Navigation
 Examples
 example_lz : Example demonstrating LZ compression and decompression
 Welcome to Oodle
 Change Log

Oodle ships with several examples.

Each file in the "examples" directory that starts with "example_" is an independent example which can be built on its own.

The Oodle examples are all console applications. So on Windows with MSVC you should create an empty Win32 console app project to get started.

To compile the examples, set your include path so that Oodle sdk include directory is in the path (so that "oodle.h" can be found).

To link the examples, you must specify one of the oodle libs. See About Oodle on Platforms for the name of the Oodle lib on your platform.

To run the examples on Windows, they must be able to find the Oodle dll (distributed in the "redist" directory). You can either run the examples in the redist directory, or copy the Oodle dll do the directory you wish to run the examples from. It is not advised to copy the Oodle dll into the windows system directory.

The Oodle examples write log files and threadprofile logs to a directory called "c:\oodlelogs" ; this directory will be created if it does not exist. If you want them to write to another place (or not write any logs) you can change the example code.

The examples use some default file names to find some data to work on; the content of these files is not important and they are not distributed with Oodle. You can copy in any file of your choice to serve as test data for the examples. Some of the examples load a file named "oodle_example_input_file" ; if this file is not found, they will create one. You can also put your own test file in its place.

Try starting with example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core
 

ExamplesExamplesexample_lz : Example demonstrating LZ compression and decompression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_FillSeekTable
Navigation
 OodleLZ_GetSeekTableMemorySizeNeeded
 OodleLZ_CreateSeekTable
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_FillSeekTable( OodleLZ_SeekTable * pTable,
                               OodleLZSeekTable_Flags flags,
                               OO_S32 seekChunkLen,
                               const void * rawBuf,
                               OO_SINTa rawLen,
                               const void * compBuf,
                               OO_SINTa compLen );
Discussion
scan compressed LZ stream to fill the seek table
Parameters
pTable  pointer to table to be filled
flags  options
seekChunkLen  the length of a seek chunk (eg from OodleLZ_MakeSeekChunkLen)
rawBuf  (optional) uncompressed buffer; used to compute the rawCRCs member of OodleLZ_SeekTable
rawLen  size of rawBuf
compBuf  compressed buffer
compLen  size of compBuf
Return Value
return  true for success
Discussion

pTable must be able to hold at least OodleLZ_GetSeekTableMemorySizeNeeded

seekChunkLen must be a multiple of OODLELZ_BLOCK_LEN. seekChunkLen must match what was in CompressOptions when the buffer was made, or any integer multiple thereof.
 

OodleLZ_GetSeekTableMemorySizeNeededOodleAPI_LZ_CompressorsOodleLZ_CreateSeekTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Compression Scratch Memory
Navigation
 About Oodle Job Threading Plugins
 About Oodle
 Welcome to Oodle
 Change Log

About Compression Scratch Memory

OodleLZ_Compress takes a "scratch memory" argument that lets the user provided a memory buffer that the compressors can use instead of allocating their own using allocation callback.

This has long been used at the lower compression levels that have fairly predictable memory usage. Starting with Oodle 2.9.9, Optimal1 and higher levels (which use more complicated data structures that have less predictable mmeory usage) use scratch memory much more extensively as well, which can greatly reduce the number of large allocations performed by the encoder.

In the common scenario where Oodle Data is used to compress individual chunks of <=256KB, allocations can often be avoided completely provided a large enough buffer is provided. For <=256KB chunks, 32MB is usually more than sufficient for all codecs and all compression levels. (Larger inputs or threaded/parallel encoding use cases are more complicated and will, in general, still do allocations.) If the scratch memory is insufficient, Oodle will use regular allocations and frees once that memory area is exhausted.

The main advantage of providing scratch memory instead of letting the encoder do its own allocations is that this allows a single large allocation to be shared over many consecutive encodes, reduction memory allocation/freeing overhead significantly. Optimal1 and higher tend to allocate a few big data structures, and these large allocations tend to have fairly high overhead otherwise.

For the intended use case (encoding lots of small independent chunks), we've seen encoding speed-ups of around 10% from using scratch memory when using a single thread. With many threads, the impact can be massive. On 64-core machines running Windows (as of Jan 2023), we've seen speed-ups of over 8x from providing scratch memory instead of using the default Windows heap, because otherwise large allocations get passed through to the kernel which then eventually serializes on page table updates when the newly allocated memory is first accessed.
 

About Oodle Job Threading PluginsAbout Oodle 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_CheckVersion
Navigation
 Oodle_SetUsageWarnings
 Oodle_LogHeader
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL Oodle_CheckVersion( OO_U32 oodle_header_version,
                            OO_U32 * pOodleLibVersion OODEFAULT( NULL ) );
Discussion
Check the Oodle lib version against the header you are compiling with
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
pOodleLibVersion  (optional) filled with the Oodle lib version
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

If you use the Oodle2 Ext lib,, OodleX_Init does it for you. But if you want to check that you have a compatible lib before trying to Init, then use this.
 

Oodle_SetUsageWarningsCore BaseOodle_LogHeader

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_Create
Navigation
 OodleLZ_Decompress
 OodleLZDecoder_MemorySizeNeeded
 Welcome to Oodle
 Change Log
// Function prototype:
OodleLZDecoder * OodleLZDecoder_Create( OodleLZ_Compressor compressor,
                                        OO_S64 rawLen,
                                        void * memory,
                                        OO_SINTa memorySize );
Discussion
Create a OodleLZDecoder
Parameters
compressor  the type of data you will decode; use OodleLZ_Compressor_Invalid if unknown
rawLen  total raw bytes of the decode (or <= 0 for any/unknown, but Reset for each buffer)
memory  (optional) provide memory for the OodleLZDecoder object (not the window)
memorySize  (optional) if memory is provided, this is its size in bytes
Return Value
return  the OodleLZDecoder
Discussion

If memory is provided, it must be of size OodleLZDecoder_MemorySizeNeeded. If it is NULL it will be allocated with the malloc specified by Core plugins.

Free with OodleLZDecoder_Destroy. You should Destroy even if you passed in the memory.

Providing compressor lets the OodleLZDecoder be the minimum size needed for that type of data. If you pass OodleLZ_Compressor_Invalid, then any type of data may be decoded, and the Decoder is allocated large enought to handle any of them.

If you are going to pass rawLen to OodleLZDecoder_Reset , then you can pass 0 to rawLen here. To make a Decoder to use with many buffer sizes, pass either <= 0 (for infinite) or the largest buffer size you can see. Then call Reset() with the correct buffer size before starting to decode each buffer.

See OodleLZDecoder_DecodeSome for more.
 

OodleLZ_DecompressOodleAPI_LZ_CompressorsOodleLZDecoder_MemorySizeNeeded

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle2 Core API Documentation
Navigation
 Welcome to Oodle
 Change Log
  •  Core Base
    •  Defines
      •  OODLE_MALLOC_MINIMUM_ALIGNMENT
      •  OODLE_JOB_MAX_DEPENDENCIES
      •  OODLE_JOB_NULL_HANDLE
      •  t_fp_Oodle_Job
      •  OODLE_HEADER_VERSION
      •  OodleNetworkVersion
    •  Enumerants
      •  Oodle_UsageWarnings
    •  Structures
      •  OodleConfigValues
    •  Functions
      •  Oodle_GetConfigValues
      •  Oodle_SetConfigValues
      •  Oodle_SetUsageWarnings
      •  Oodle_CheckVersion
      •  Oodle_LogHeader
    •  Typedefs
      •  t_OodleFPVoidVoid
      •  t_OodleFPVoidVoidStar
  •  Core plugins
    •  Functions
      •  OodleCore_Plugins_SetAllocators
      •  OodleCore_Plugins_SetJobSystem
      •  OodleCore_Plugins_SetJobSystemAndCount
      •  OodleCore_Plugins_SetPrintf
      •  OodleCore_Plugins_SetAssertion
    •  Typedefs
      •  t_fp_OodleCore_Plugin_MallocAligned
      •  t_fp_OodleCore_Plugin_Free
      •  t_fp_OodleCore_Plugin_RunJob
      •  t_fp_OodleCore_Plugin_WaitJob
      •  t_fp_OodleCore_Plugin_Printf
      •  t_fp_OodleCore_Plugin_DisplayAssertion
  •  Core LZ compression
    •  About OodleLZ
    •  About OodleLZ ThreadPhased Decode
    •  About OodleLZ Hydra
    •  OodleAPI_LZ_Compressors
      •  Defines
        •  OODLE_ALLOW_DEPRECATED_COMPRESSORS
        •  OODLELZ_LOCALDICTIONARYSIZE_MAX
        •  OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
        •  OODLELZ_BLOCK_LEN
        •  OODLELZ_BLOCK_MAX_COMPLEN
        •  OODLELZ_QUANTUM_LEN
        •  OODLELZ_FAILED
        •  OODLELZ_SCRATCH_MEM_NO_BOUND
      •  Enumerants
        •  OodleLZ_Verbosity
        •  OodleLZ_Compressor
        •  OodleLZ_PackedRawOverlap
        •  OodleLZ_CheckCRC
        •  OodleLZ_Profile
        •  OodleDecompressCallbackRet
        •  OodleLZ_CompressionLevel
        •  OodleLZ_Jobify
        •  OodleLZ_Decode_ThreadPhase
        •  OodleLZ_FuzzSafe
        •  OodleLZSeekTable_Flags
        •  OodleLZ_CompressScratchMemBoundType
      •  Structures
        •  OodleLZ_CompressOptions
        •  OodleLZ_DecodeSome_Out
        •  OodleLZDecoder
        •  OodleLZ_SeekTable
      •  Functions
        •  OodleLZ_Compress
        •  OodleLZ_Decompress
        •  OodleLZDecoder_Create
        •  OodleLZDecoder_MemorySizeNeeded
        •  OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
        •  OodleLZDecoder_Destroy
        •  OodleLZDecoder_Reset
        •  OodleLZDecoder_DecodeSome
        •  OodleLZDecoder_MakeValidCircularWindowSize
        •  OodleLZ_MakeSeekChunkLen
        •  OodleLZ_GetNumSeekChunks
        •  OodleLZ_GetSeekTableMemorySizeNeeded
        •  OodleLZ_FillSeekTable
        •  OodleLZ_CreateSeekTable
        •  OodleLZ_FreeSeekTable
        •  OodleLZ_CheckSeekTableCRCs
        •  OodleLZ_FindSeekEntry
        •  OodleLZ_GetSeekEntryPackedPos
        •  OodleLZ_CompressionLevel_GetName
        •  OodleLZ_Compressor_GetName
        •  OodleLZ_Jobify_GetName
        •  OodleLZ_CompressOptions_GetDefault
        •  OodleLZ_CompressOptions_Validate
        •  OodleLZ_Compressor_UsesWholeBlockQuantum
        •  OodleLZ_Compressor_UsesLargeWindow
        •  OodleLZ_Compressor_CanDecodeInCircularWindow
        •  OodleLZ_Compressor_CanDecodeThreadPhased
        •  OodleLZ_Compressor_CanDecodeInPlace
        •  OodleLZ_Compressor_MustDecodeWithoutResets
        •  OodleLZ_Compressor_CanDecodeFuzzSafe
        •  OodleLZ_Compressor_RespectsDictionarySize
        •  OodleLZ_GetCompressScratchMemBound
        •  OodleLZ_GetCompressScratchMemBoundEx
        •  OodleLZ_GetCompressedBufferSizeNeeded
        •  OodleLZ_GetDecodeBufferSize
        •  OodleLZ_GetInPlaceDecodeBufferSize
        •  OodleLZ_GetCompressedStepForRawStep
        •  OodleLZ_GetAllChunksCompressor
        •  OodleLZ_GetFirstChunkCompressor
        •  OodleLZ_GetChunkCompressor
      •  Typedefs
        •  OodleDecompressCallback

 
  Core Base

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILEINFO_MODTIME_INVALID
Navigation
 OODLEX_FILEINFO_FLAG_INVALID
 OODLEX_FILE_SIZE_INVALID
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_FILEINFO_MODTIME_INVALID ((OO_U64)-1)
Discussion
Invalid value for OodleXFileInfo:modTime
 
OODLEX_FILEINFO_FLAG_INVALIDOodleX low level async ioOODLEX_FILE_SIZE_INVALID

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_SCRATCH_MEM_NO_BOUND
Navigation
 OODLELZ_FAILED
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_SCRATCH_MEM_NO_BOUND (-1)
Discussion
Scratch mem size when bound is unknown. Installed allocator may be used no matter how much scratch mem you provide.
 
OODLELZ_FAILEDOodleAPI_LZ_CompressorsOodleLZ_Verbosity

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_SelectDictionaryFromPackets_Trials
Navigation
 OodleNetwork1_SelectDictionaryFromPackets
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1_SelectDictionaryFromPackets_Trials( void * dictionary_to_fill,
                                                          OO_S32 dictionary_size,
                                                          OO_S32 htbits,
                                                          const void * * dictionary_packet_pointers,
                                                          const OO_S32 * dictionary_packet_sizes,
                                                          OO_S32 num_dictionary_packets,
                                                          const void * * test_packet_pointers,
                                                          const OO_S32 * test_packet_sizes,
                                                          OO_S32 num_test_packets,
                                                          OO_S32 num_trials,
                                                          double randomness_percent,
                                                          OO_S32 num_generations );
Discussion
Multi-Trial variant of OodleNetwork1_SelectDictionaryFromPackets
Parameters
num_trials  number of trials per generation; 5-20 is a good starting range
randomness_percent  randomness of trials; this is a percent of standard, 100 is a good default; 50-200 is a useful range
num_generations  number of generations; 1 is fine, more is slower
Return Value
return  true on success, false otherwise.
Discussion

This function runs the packet selector of OodleNetwork1_SelectDictionaryFromPackets repeatedly with some randomness. Variation of packet selection can sometimes give slightly better dictionaries.

The success of the trial is measured on the "test_packet" set. Make sure that the "dictionary_packet" set are independently randomly drawn from the packet source data wrst the "test_packet" set so that over-training degeneracies are not created.

On platforms where OodleNetwork1_SelectDictionarySupported returns false, this function is not supported.

It's advised to run this function in 64-bit, as it can use a lot of memory.
 

OodleNetwork1_SelectDictionaryFromPacketsOodleAPI_OodleNetwork1About Oodle Network Compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_Printf
Navigation
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OodleXLog_Printf OodleXLog_Printf_Raw(verboseLevel,__FILE__,__LINE__,##__VA_ARGS__)
Discussion
OodleXLog_Printf lets you write to Oodle's log.
Discussion
Use like printf : OodleXLog_Printf(verbose,fmt,arg1,arg2,...)

What kind of output is produced from this depends on the bit flags set in OodleXLog_SetState.

If the global verbose level set by OodleXLog_SetVerboseLevel is < verboseLevel passed here, the message is supressed.

OodleXLog_Printf_vN(fmt,...) is the same as OodleXLog_Printf(N,fmt,...)
 

OodleX Debug aidsOodleX Debug aidsOodleXLog_StateFlags

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_ASYNC_HANDLE_ERROR
Navigation
 OODLEX_ASYNC_HANDLE_DONE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_ASYNC_HANDLE_ERROR ((OodleXHandle)0x0000000200000001ULL)
Discussion
OodleXHandle to a special always-error handle. Calls to OodleX_GetStatus on this handle value will return &OodleXStatus_Error. This handle must not be deleted! Do not call OodleX_Wait on it with deleteIfDone = true.
 
OODLEX_ASYNC_HANDLE_DONEOodleX async handle operationsOodleXPriority

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ReadMallocWholeFile_GetResult
Navigation
 OodleXIOQ_ReadMallocWholeFile_Async
 OodleXIOQ_OpenAndReadMallocWholeFile_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXIOQ_ReadMallocWholeFile_GetResult( OodleXHandle req,
                                                      OodleXHandleDeleteIfDone andDeleteIfDone,
                                                      void * * pPtr,
                                                      OO_S64 * pSize OODEFAULT( NULL ) );
Discussion
Finish a OodleXIOQ_ReadMallocWholeFile_Async request
Parameters
req  the OodleXHandle to the OodleXIOQ_ReadMallocWholeFile_Async request
andDeleteIfDone  if true and the returned status is >= Done the handle will be deleted
pPtr  filled out with the buffer allocated by OodleXIOQ_ReadMallocWholeFile_Async
pSize  (optional) filled with the file size
Return Value
return  the status ; if <= OodleXStatus_Pending , result pointers are set to null
Discussion

OodleXIOQ_ReadMallocWholeFile_GetResult does NOT wait on the handle.

See OodleXIOQ_ReadMallocWholeFile_Async
 

OodleXIOQ_ReadMallocWholeFile_AsyncOodleX low level async ioOodleXIOQ_OpenAndReadMallocWholeFile_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Mac
Navigation
 About Oodle on Nintendo Switch
 About Oodle on Platforms
 About Oodle on Xbox One
 Welcome to Oodle
 Change Log

Oodle2 for Mac is provided as a shared library. (change in 2.5.1 : static libraries now provided as well)

lib/liboo2coremac.version.dylib
lib/liboo2coremac64.version.dylib
lib/liboo2extmac.version.dylib
lib/liboo2extmac64.version.dylib

Where "version" is the current Oodle version (2.9.11).

Oodle for Mac is provided in 32 and 64 bit builds.

The debug build of the Oodle lib is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems.


OodleX for Mac tries to write a log file to "/var/log/oodle/" by default. If you don't want this, you may disable logging or change the log location in OodleXInitOptions. On most systems, Oodle will fail to create the "oodle" subdir in "var/log/" due to lack of permissions; you must create that dir for Oodle. When Oodle fails to write the log to the desired location, it will instead write it to "." (current directory).
 

About Oodle on Nintendo SwitchAbout Oodle on PlatformsAbout Oodle on Xbox One

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_WORKERS_COUNT_ALL_HYPER_CORES
Navigation
 OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_WORKERS_COUNT_ALL_HYPER_CORES (-2)
Discussion
Make workers for every hyper-thread eg. in a 6-physical core, 12-hyper-thread system, would make 12 threads this is usually best for Oodle Texture work See also OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
 
OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORESOodleX Startup and ShutdownOodleX_Init_GetDefaults_DebugSystems

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetErrorDetails
Navigation
 OodleXIOQ_GetStatus
 OodleXIOQ_GetErrorEnum
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_GetErrorDetails( OO_U32 code,
                                   OodleXIOQFile file,
                                   char * pMessage,
                                   int messageSize );
Discussion
Convert an OS error code into a text message
Parameters
code  the error code, eg. from OodleXIOQ_GetStatus
file  the file that the error occurred on (or 0 for unknown)
pMessage  pointer to a text buffer that will be filled out
messageSize  number of bytes in the text buffer
Return Value
return  bool for success/failure
Discussion

fills out pMessage with a text description of the error (if available).
 

OodleXIOQ_GetStatusOodleX low level async ioOodleXIOQ_GetErrorEnum

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_Oodle_Job
Navigation
 OODLE_JOB_NULL_HANDLE
 OODLE_HEADER_VERSION
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define t_fp_Oodle_Job t_OodleFPVoidVoidStar
Discussion
Job function pointer for Plugin Jobify system
Discussion
takes void pointer returns void
 
OODLE_JOB_NULL_HANDLECore BaseOODLE_HEADER_VERSION

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandle
Navigation
 Welcome to Oodle
 Change Log
// Type definition:
OO_U64 OodleXHandle
Discussion
Opaque weak reference to Oodle asynchronous objects
Discussion

Any op which returns an OodleXHandle can be used in OodleX_Wait or as a dependency for other ops.

See OodleX async handle operations
 

OodleXHandleCountdown_DecrementOodleX async handle operationsOodleX threading util

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How do I use Oodle with no allocator?
Navigation
 FAQ: OodleLZ_Decompress is failing how do I diagnose it?
 Frequently Asked Questions
 FAQ: How do I limit the encoder memory use?
 Welcome to Oodle
 Change Log

If you want to use Oodle2 Core without it doing any allocations itself, you may still do decompression.

1. Don't use OodleX. Use Oodle Core only. OodleX requires allocators and installs its own.

2. The default allocator in Oodle Core is clib. If you want to ensure the clib allocators are never used, you may call OodleCore_Plugins_SetAllocators with NULL function pointers to ensure all allocations are forbidden. Note that if you call anything in Oodle that attempts to allocate, it will be a fatal error. For production it's safer to just leave the clib allocator in place; try setting the allocators to NULL only in a developer's debug build to verify that Oodle is not calling the allocator. (See example_lz_noallocs : Example demonstrating Oodle compression with no allocations)

3. To do LZ decoding, call OodleLZDecoder_Create or OodleLZ_Decompress, and provide pre-allocated memory for the decoder with the decoderMemory argument, of size OodleLZDecoder_MemorySizeNeeded.

4. To do LZ encoding, call OodleLZ_Compress with scratchMem large enough for all the encoder's memory needs. Call OodleLZ_GetCompressScratchMemBound to find the size needed. If the scratchMem you provide is not big enough, the encoder will use the installed allocator function (and stop the process if you made it null). Note that only the new LZ's and levels equal to or below "Normal" can be made allocation free, the old compressors and Optimal levels cannot.

5. You can also do OodleNetwork1 network packet encoding and decoding from previously trained states without Oodle doing any internal allocations (you provide all required memory). You cannot do the off-line training or dictionary selection, those require an allocator.

6. The encoder & decoder only use the provided memory as scratch space, not as retained data storage. This means you can reuse it between calls, it can be allocated only once and used as scratch each time. Note however that each thread must have its own scratch memory, it cannot be used simultaneously by multiple threads.

7. See example_lz_noallocs : Example demonstrating Oodle compression with no allocations
 

FAQ: OodleLZ_Decompress is failing how do I diagnose it?Frequently Asked QuestionsFAQ: How do I limit the encoder memory use?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_LogHeader
Navigation
 Oodle_CheckVersion
 Welcome to Oodle
 Change Log
// Function prototype:
void Oodle_LogHeader( void );
Discussion
Log the Oodle version & copyright
Discussion
Uses the log set with OodleCore_Plugins_SetPrintf
 
Oodle_CheckVersionCore Baset_OodleFPVoidVoid

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_RespectsDictionarySize
Navigation
 OodleLZ_Compressor_CanDecodeFuzzSafe
 OodleLZ_GetCompressScratchMemBound
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_RespectsDictionarySize( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor obeys OodleLZ_CompressOptions:dictionarySize which limits match references to a finite bound. (eg. for sliding window decompression).

All the new codecs do (Kraken,Mermaid,Selkie,Leviathan). Some old codecs don't.
 

OodleLZ_Compressor_CanDecodeFuzzSafeOodleAPI_LZ_CompressorsOodleLZ_GetCompressScratchMemBound

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Read_Async
Navigation
 OodleXIOQ_CloseFileRename_Async
 OodleXIOQ_Write_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_Read_Async( OodleXIOQFile file,
                                   void * memory,
                                   OO_SINTa size,
                                   OO_S64 position,
                                   OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                   OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                   const OodleXHandle * dependencies OODEFAULT( NULL ),
                                   OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a read request
Parameters
file  the file to act on
memory  memory to read into (must be OODLEX_IO_MAX_ALIGNMENT aligned)
size  number of bytes to read (must be OODLEX_IO_MAX_ALIGNMENT aligned)
position  file position to start the read (must be OODLEX_IO_MAX_ALIGNMENT aligned)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Starts an async read into memory at file offset _position , of _size bytes.

To do unaligned reads, use OodleXIOQ_ReadUnalignedAdjustPointer_Async , or simply read a larger amount, and use OodleX_IOAlignDownS64 on position and OodleX_IOAlignUpS64 on _size.

The read is not done when OodleXIOQ_Read_Async returns. You must not free memory until the read is done, as reported by the handle returned;
 

OodleXIOQ_CloseFileRename_AsyncOodleX low level async ioOodleXIOQ_Write_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
Navigation
 OodleXIOQ_OpenWriteWholeFileClose_Async
 OodleXIOQ_ReadUnalignedAdjustPointer_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_OpenWriteWholeFileCloseTempName_Async( const char * name,
                                                              const void * buffer,
                                                              OO_SINTa size,
                                                              OodleXFileOpenFlags fileOpenFlags OODEFAULT( OodleXFileOpenFlags_Default ),
                                                              const OodleXFileOpsVTable * vtable OODEFAULT( NULL ),
                                                              OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                              OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                              const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                              OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to open a file, write a buffer, close it, and rename it.
Parameters
name  name of the file to open (VFS, UTF-8)
buffer  the buffer to write (must be OODLEX_IO_MAX_ALIGNMENT aligned)
size  the final file size (no alignment required)
fileOpenFlags  (optional) flags for the os file open (see OodleXFileOpenFlags)
vtable  (optional) the OodleXFileOpsVTable to use for all ops on this file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

This is the same as OodleXIOQ_OpenWriteWholeFileClose_Async , but the writing is done to a temp file, and then renamed to name at the end, like OodleXIOQ_CloseFileRename_Async. The rename is only done if the writing succeeded.


 

OodleXIOQ_OpenWriteWholeFileClose_AsyncOodleX low level async ioOodleXIOQ_ReadUnalignedAdjustPointer_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignUpS64
Navigation
 OodleX_IOAlignUpS32
 OodleX_IOAlignUpSINTa
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S64 OodleX_IOAlignUpS64( const OO_S64 x );
Discussion
Align up to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x up to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_IOAlignUpS32OodleX UtilsOodleX_IOAlignUpSINTa

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleFileNotFoundIsAnError
Navigation
 OodleXCopyFileFlags
 OODLEX_FILEINFO_FLAGS
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleFileNotFoundIsAnError
{
    OodleFileNotFoundIsAnError_No = 0,
    OodleFileNotFoundIsAnError_Yes = 1,
    OodleFileNotFoundIsAnError_Force32 = 0x40000000
};
Discussion
Bool for whether a file not found is a completion status of OodleXStatus_Error or OodleXStatus_Done
 
OodleXCopyFileFlagsOodleX low level async ioOODLEX_FILEINFO_FLAGS

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ClearError
Navigation
 OodleXIOQ_GetLastError
 OodleXIOQ_LogLastError
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXIOQ_ClearError( OodleXIOQFile file );
Discussion
Clear any errors on the file
Parameters
file  the IOQFile to query
Discussion

Wipe out any previous errors recorded on the file, so that OodleXIOQ_GetLastError now returns zero.
 

OodleXIOQ_GetLastErrorOodleX low level async ioOodleXIOQ_LogLastError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle Job Threading Plugins
Navigation
 About Oodle on Platforms
 About Oodle
 About Compression Scratch Memory
 Welcome to Oodle
 Change Log

About Oodle Job Threading Plugins

Oodle Core can run multi-threaded with a user-supplied job system via OodleCore_Plugins_SetJobSystem

Oodle Core does not provide its own threading (or IO). It is as low level and minimal as possible, doing primary memory to memory compression and decompression. High level and system-dependent functionality is kept in OodleX.

If you use OodleX, it plugs in its own job system to Oodle Core automatically in OodleX_Init.

If you do not use OodleX, you may plug in your own job system to get threading of Oodle Core. Some examples are provided of common job systems (see example_jobify_gcd.mm , example_jobify_linuxtbb.cpp, or example_jobify_win32tp.cpp).

Currently Oodle Core uses the Job plugin system to parallelize Optimal level encoding in OodleLZ_Compress. It is controlled via the OodleLZ_CompressOptions:jobify option (see OodleLZ_Jobify).

The default setting of OodleLZ_CompressOptions:jobify, OodleLZ_Jobify_Default, is to use a job system for multithreading if one is provided.

When a job system is provided, encoding of buffers larger than one OODLELZ_BLOCK_LEN (256 KB), at OodleLZ_CompressionLevel greater or equal to OodleLZ_CompressionLevel_Optimal1, will use multiple threads.

If you are encoding small chunks (less or equal to OODLELZ_BLOCK_LEN), or if you are already encoding lots of files at the same time with multiple threads, or if you are encoding below level Optimal1, Jobify will not do much for you. If you are encoding only one file at a time (or care about latency rather than throughput), if you use level >= Optimal1, and you encode large files, Jobify will improve encode speed.

Jobify summary :

  • What is currently threaded? Only encoding (OodleLZ_Compress), of the new LZ codecs (Kraken,Mermaid,Selkie,Leviathan), at level Optimal1+ , only with more than 1 BLOCK

  • Jobify does nothing if you encode single BLOCKS or smaller (256 KB)

  • if you parallel encode many files at a time (which you should), further encoder threading with Jobify will be a big benefit (but also doesn't hurt)

  • Jobify uses Wait() on workers from the calling thread. if your calling thread is already a worker, you must be able to wait-from-worker which many thread systems cannot do

  • Jobify default (OodleLZ_Jobify_Default) is "yes do jobify if a thread system is provided" so you do not need to set any options to get threaded encoding enabled. If you want to turn off threaded encoding, even though you provided a thread system, set OodleLZ_CompressOptions:jobify to OodleLZ_Jobify_Disable.

  • OodleX provides its thread system automatically, so if you use OodleX then jobify will just be on by default without you doing anything

  • example plugin job system implementations are provided (see example_jobify_gcd.mm , example_jobify_linuxtbb.cpp, or example_jobify_win32tp.cpp).

  • Note that when possible you should paralellelize at the chunk level or the file level. This is provides better parallelism than Jobify.

If you provide your own plugins for Jobify, be aware of the requirements on RunJob and WaitJob :

  • RunJob returns an async handle that can be passed to WaitJob. That handle should be unique until it is returned by the paired call to WaitJob.

  • RunJob may be passed dependencies. It should not start its job function until those depenencies are done, but this is not the same as WaitJob - should not delete those dependency jobs.

  • WaitJob may be called before or after the job is done. It should wait on the job (if it is still running), then clean it up and release the job handle.

  • Job handle value 0 is reserved to mean "none"; if RunJob returns 0 it means the work was done synchronously.

  • WaitJob will only be called from the original thread that called OodleLZ_Compress. It will not be called from a job function we invoked.

Some thread systems cannot wait from a worker thread. If you are running OodleLZ_Compress from a worker thread, be aware then WaitJob may be called on that thread. If Jobify uses the same thread system, WaitJob can be a deadlock.

One option is to use a separate thread pool for your top level parallelism (parallelizing on files or chunks), and a separate pool for Jobify. That way thread pool 1 only waits on thread pool 2. There is no wait-on-workers from within the same pool.

OodleX worker thread system supports wait-from-worker so it is okay to use Jobify and OodleLZ parallelism For example, if you call OodleXLZ_Compress_Async with OodleLZ_CompressOptions:seekChunkReset, it will parallellize at the seek chunk level, and then again at the jobify level (if each seek chunk is bigger than 1 block). This means the jobify-level parallelism will call WaitJob on the worker threads that were spawned for seek-level parallelism. This may not work in some thread systems but is okay with OodleX.

(note that OodleX supports wait-from-worker only for Jobs; you should not OodleX_Wait on non-Job handles from inside a Job, but you can use them as dependencies)
 

About Oodle on WASMAbout OodleAbout Compression Scratch Memory

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_GetEcho
Navigation
 OodleXLog_SetEcho
 OodleXLog_SetCallback
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXLog_GetEcho( );
Discussion
Get the current echo FILE
Return Value
return  the current echo FILE
Discussion

See OodleXLog_SetEcho
 

OodleXLog_SetEchoOodleX Debug aidsOodleXLog_SetCallback

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetLastError
Navigation
 OodleXIOQ_GetName
 OodleXIOQ_ClearError
 Welcome to Oodle
 Change Log
// Function prototype:
OO_U32 OodleXIOQ_GetLastError( OodleXIOQFile file );
Discussion
Get the last error on a file
Parameters
file  the IOQFile to query
Return Value
return  the last error on the file (0 for none)
Discussion

IO operation errors are tracked on the file to simplify error tracking. Individual operation errors can be queried with OodleXIOQ_GetStatus. The error code returned can be processed with OodleXIOQ_GetErrorEnum or OodleXIOQ_GetErrorDetails.
 

OodleXIOQ_GetNameOodleX low level async ioOodleXIOQ_ClearError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Getting Started with Oodle LZ Data Compression
Navigation
 Welcome to Oodle
 Change Log

Oodle compresses generic data with lossless (LZ) data compression. Oodle provides many different levels of speed vs compression ratio tradeoffs. Oodle lets you use the same data compression and packaging formats on all platforms. (see About OodleLZ for more)

If you're benchmarking Oodle compression, please consider Tips for benchmarking a compressor


To sample a range of Oodle compressor speeds & compression ratios, try example_lz_chart : Example that makes a chart of OodleLZ options.

example_lz_chart is now provided as a pre-built executable on desktop platforms. Just go run it on one of your data files to get a sample of Oodle's performance.

The compressors offer a variety of tradeoffs to suit different needs (see About OodleLZ). The higher CompressionLevel always provides smaller file sizes at the expense of longer encode times.

You can also use the "ozip" command line utility to test Oodle without building any code. ozip lets you try various Oodle options and test compression ratios and performance with the ozip -b benchmark mode. See About Oodle ozip.

Feel free to contact oodle@radgametools.com for help on deciding which compressor is best for your needs.


Try building an example. example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core is a good place to start. It includes only "oodle2.h" from the include dir. Link it with the appropriate oo2core lib from the "lib" directory. (on Windows, you also need the DLL from the "redist" directory).

Now run it on one of your game data files to try it.

example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core uses the compressor OodleLZ_Compressor_Kraken , and the encode effort OodleLZ_CompressionLevel_Normal. Kraken generally provides a good tradeoff between compression ratio and decode speed. A higher CompressionLevel trades off encode speed for greater compression ratio.


Using Oodle for simple memory to memory compression

To compress memory->memory synchronously, you call OodleLZ_Compress. To decompress the same way you call OodleLZ_Decompress. These are in the API group OodleAPI_LZ_Compressors in Oodle Core.

To decode parts of the stream into a buffer incrementally use OodleLZDecoder_DecodeSome

Let's walk through a simple memory to memory encode and decode with Oodle's LZ compressors. The LZ compressors are lossless and compress any kind of data.

For this simple synchronous operation, we can use just Oodle Core, not OodleX. No intialization is required.

We can simply make the call to do memory->memory compression :


	SINTa comp_len = OodleLZ_Compress(OodleLZ_Compressor_Kraken,raw_buf,raw_len,comp_buf,
		OodleLZ_CompressionLevel_Optimal1);

where comp_buf should be at least OodleLZ_GetCompressedBufferSizeNeeded bytes large to take the compressed data.

Here we have chosen the compressor OodleLZ_Compressor_Kraken , which is generally a good balance of compression ratio and decode speed. We've also chosen a high compression level (OodleLZ_CompressionLevel_Optimal1), so this encode will be rather slow, but give us a smaller output size. The compression level sets the amount of effort the encoder does for string matching, and is how you can control the encode time (try OodleLZ_CompressionLevel_Fast for example). The compression level does not affect the decode time, generally.

You can then decompress in a similar way :


	SINTa dec_len = OodleLZ_Decompress(comp_buf,comp_len,dec_buf,raw_len);
	ASSERT( dec_len == raw_len );

OodleLZ_Decompress will decode data from any of the compressors. Raw OodleLZ compressed data is headerless - you have to pass in the raw and compressed data sizes to the decode call. See About OodleLZ for more details about OodleLZ data.

Oodle Core requires no shutdown.

You can see a full working example in example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core.


A note on evaluating Oodle

When evaluating Oodle, be careful what kind of data you use to test the compressors.

You should not run the compressors on already-compressed data such as PNG or JPEG images or Bink movies. No compressor can do much with already-compressed data, so the differences between them will be hidden.

If you have your game data in an existing archive like a Zip, unpack that before trying those files in Oodle.

Oodle is optimized for typical game data - levels, meshes, animations, bitmap and hardware compressed textures.

To use Oodle with Granny animation data, make sure you disable Granny's built-in compressor on those files.

In general, on signal media such as images, audio and video, the appropriate data-specific compressor should be used. On other data that have optional compression, it should be turned off. For example things like pdf or geometry mesh formats that have optional zlib compression, it's better to turn that compression off and give the data uncompressed to Oodle, because Oodle will do a better job.

If your data is not compressing as you hoped, feel free to contact oodle@radgametools.com and we'll have a look at it.

Also, if you want to donate some data to go in our test set, that would be appreciated, and the Oodle compressors will get better at your data as we refine them.

See some tips on benchmarking here : Tips for benchmarking a compressor


Oodle does not provide packaging or archiving. You create your packages however you want for you game, and use Oodle just for the data compression.

OodleX provides multi-threaded encode and decode. You can also write your own parallel encode and decode using Oodle Core with your own threading system.

Oodle currently provides only generic lossless data compression, no specialized image or audio compression.


The Oodle LZ compressors can encode & decode incrementally, they can encode & decode multi-threaded, they can provide seek points for random access in compressed streams, they can overlap IO and encoding or decoding. There are lots of capabilities as you dig deeper.

You can examine some of the more advanced LZ Examples , or visit About OodleLZ or Frequently Asked Questions, or contact oodle@radgametools.com.


For a guide to how to build with the Oodle library on your platform, see About Oodle on Platforms. For information about the Core vs Ext libs see Oodle2 Core vs Oodle2 Ext.
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
Navigation
 OODLELZ_LOCALDICTIONARYSIZE_MAX
 OODLELZ_BLOCK_LEN
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT (256)
Discussion
Default value of spaceSpeedTradeoffBytes in OodleLZ_CompressOptions Changes how the encoder makes decisions in the bit stream Higher spaceSpeedTradeoffBytes favors decode speed more (larger compressed files) Lower spaceSpeedTradeoffBytes favors smaller compressed files (slower decoder) Goes in a power of 2 scale; so try 64,128 and 512,1024 (OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT/2) or (OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT*2)
 
OODLELZ_LOCALDICTIONARYSIZE_MAXOodleAPI_LZ_CompressorsOODLELZ_BLOCK_LEN

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Semaphore
Navigation
 OodleX_ThreadFunc
 Welcome to Oodle
 Change Log
// Type definition:
OO_U32 OodleX_Semaphore
Discussion
Semaphore ; initialize with = 0 , no cleanup necessary
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX_GetNumWorkerThreadsOodleX threading utilOodleX_ThreadFunc

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_GetDefaults
Navigation
 OodleX_Init_ThreadProfilerInit
 OodleX_Init_GetDefaults_Minimal
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleX_Init_GetDefaults( OO_U32 oodle_header_version,
                                 OodleXInitOptions * pOptions,
                                 OodleX_Init_GetDefaults_DebugSystems debugSystems OODEFAULT( OodleX_Init_GetDefaults_DebugSystems_Yes ),
                                 OodleX_Init_GetDefaults_Threads threads OODEFAULT( OodleX_Init_GetDefaults_Threads_Yes ) );
Discussion
Get defaults for OodleXInitOptions
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
pOptions  filled with default OodleXInitOptions
debugSystems  should OodleX_Init enable any debug systems (leaktrack, log, etc) ?
threads  should OodleX_Init start any threads?
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

The debugSystems and threads options are just easy ways of getting pOptions filled out for common use cases. For fine control of individual settings, you can always set the values in OodleXInitOptions yourself.

NOTE : do not use this if you want minimal linkage. See OodleX_Init_GetDefaults_Minimal.
 

OodleX_Init_ThreadProfilerInitOodleX Startup and ShutdownOodleX_Init_GetDefaults_Minimal

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetworkVersion
Navigation
 OODLE_HEADER_VERSION
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OodleNetworkVersion 2.9.11
Discussion


 

OODLE_HEADER_VERSIONCore BaseOodle_UsageWarnings

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetFileSize_AsyncAndWait
Navigation
 OodleXIOQ_Rename_AsyncAndWait
 OodleXIOQ_NameIsDir_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S64 OodleXIOQ_GetFileSize_AsyncAndWait( const char * vfsName,
                                           OodleFileNotFoundIsAnError fnfiae );
Discussion
Convenience version of OodleXIOQ_GetInfoByName_AsyncAndWait returns negative for error
 
OodleXIOQ_Rename_AsyncAndWaitOodleX low level async ioOodleXIOQ_NameIsDir_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_CloseFile_Async
Navigation
 OodleXIOQ_OpenForWriteTempName_Async
 OodleXIOQ_CloseFileRename_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_CloseFile_Async( OodleXIOQFile file,
                                        OO_S64 truncateFileSize OODEFAULT( OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE ),
                                        OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                        OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                        const OodleXHandle * dependencies OODEFAULT( NULL ),
                                        OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a close-file request
Parameters
file  the file to close
truncateFileSize  (optional) truncate an OpenforWrite file
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

CloseFile also copies any errors on the file to the request, so that an OodleXIOQ_GetStatus on the CloseFile request will return OodleXStatus_Error if there are any errors on the file.

If the file was OpenForWrite, then truncateFileSize can be used to set the final file size. This is mainly used when the file was reserved with OodleXIOQ_ReserveFileSizeForWrite_Async , but it should also be used any time a file size that is not OODLEX_IO_MAX_ALIGNMENT aligned is desired. truncateFileSize does not need to be OODLEX_IO_MAX_ALIGNMENT aligned, but all sized for OodleXIOQ_Write_Async do, so without doing this file sizes will be aligned up. Pass OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE (or use the default argument) if you don't want to truncate.
 

OodleXIOQ_OpenForWriteTempName_AsyncOodleX low level async ioOodleXIOQ_CloseFileRename_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Write_Async
Navigation
 OodleXIOQ_Read_Async
 OodleXIOQ_SetFileSize_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_Write_Async( OodleXIOQFile file,
                                    const void * memory,
                                    OO_SINTa size,
                                    OO_S64 position,
                                    OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                    OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                    const OodleXHandle * dependencies OODEFAULT( NULL ),
                                    OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a write request
Parameters
file  the file to act on
memory  memory to write from (must be OODLEX_IO_MAX_ALIGNMENT aligned)
size  number of bytes to write (must be OODLEX_IO_MAX_ALIGNMENT aligned)
position  file position to start the write (must be OODLEX_IO_MAX_ALIGNMENT aligned)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Starts an async write from memory at file offset _position , of _size bytes.

The write is not done when OodleXIOQ_Write_Async returns. You must not free memory until the write is done, as reported by the handle returned.

Writes are faster on some platforms if the file size is first reserved past the end of the write, using OodleXIOQ_SetFileSize_Async or OodleXIOQ_ReserveFileSizeForWrite_Async.
 

OodleXIOQ_Read_AsyncOodleX low level async ioOodleXIOQ_SetFileSize_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Delete_Async
Navigation
 OodleXIOQ_ForceWriteable_Async
 OodleXIOQ_Rename_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_Delete_Async( const char * name,
                                     OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                     OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                     const OodleXHandle * dependencies OODEFAULT( NULL ),
                                     OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a delete request.
Parameters
name  the file to delete (VFS, UTF-8)
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

Asynchronously delete a file or dir.

Use OodleXIOQ_ForceWriteable_Async before the Delete to force the deletion of read-only and other no-access conditions.
 

OodleXIOQ_ForceWriteable_AsyncOodleX low level async ioOodleXIOQ_Rename_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_WaitDoneAllPending
Navigation
 OodleXIOQ_KickAnyDelayed
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXIOQ_WaitDoneAllPending( );
Discussion
Block the calling thread until all pending IOQ operations are complete
Discussion
Should generally only be used for errors or shutdown. OodleX_WaitDoneAllPending does this and more.
 
OodleXFileInfoOodleX low level async ioOodleXIOQ_KickAnyDelayed

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Wait
Navigation
 OodleXIOQ_LogError
 OodleXIOQ_GetInfo
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXIOQ_Wait( OodleXHandle req,
                             OodleXHandleDeleteIfDone andDelete,
                             OO_U32 * pErrorCode OODEFAULT( NULL ) );
Discussion
Block the calling thread until request is not pending
Parameters
req  the IOQ operation handle to work on
andDelete  if true, delete the request
pErrorCode  (optional) filled with the os error code, if any
Return Value
return  the status
Discussion

The status returned will not be OodleAsync_Pending. Similar to OodleX_Wait , but only works on IOQ requests, and can return the IOQ error code. Generally you should just call OodleX_Wait in most cases.
 

OodleXIOQ_LogErrorOodleX low level async ioOodleXIOQ_GetInfo

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_SetState
Navigation
 OodleXLog_SetEcho
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXLog_SetState( OO_U32 options );
Discussion
Set the global log state
Parameters
options  a combination of OodleXLog_StateFlags
Discussion

SetState(0) disables all logging. Combine the bit mask flags of OodleXLog_StateFlags with logical OR to create a log state.
 

OodleXLogCallbackRetRetOodleX Debug aidsOodleXLog_SetEcho

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_Train
Navigation
 OodleNetwork1TCP_State_InitAsCopy
 OodleNetwork1TCP_Encode
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNetwork1TCP_Train( OodleNetwork1TCP_State * state,
                             const OodleNetwork1_Shared * shared,
                             const void * * training_packet_pointers,
                             const OO_S32 * training_packet_sizes,
                             OO_S32 num_training_packets );
Discussion
Fill a OodleNetwork1TCP_State from training data
Parameters
state  the OodleNetwork1TCP_State which is filled out; this state should not need to be initialized in any way before calling Train, it will be reset internally.
shared  the OodleNetwork1_Shared data to use in compression ; this shared data should already have had OodleNetwork1_Shared_SetWindow done on it.
training_packet_pointers  array of pointers to packet data; array of size num_training_packets
training_packet_sizes  array of sizes of packets; array of size num_training_packets
num_training_packets  number of packets
Discussion

OodleNetwork1TCP_Train uses the provided training packet data to initialize state.

The training packet data provided here should not overlap the window passed to OodleNetwork1_Shared_SetWindow ; it should not come from the same source or you will get false training.

You may call OodleNetwork1_Shared_SetWindow and OodleNetwork1TCP_Train many times with different windows to optimize the window selection.

Once training is done, the resulting OodleNetwork1TCP_State should be written to disk and used by both the client and server as the initial channel state in OodleNetwork1TCP_State_InitAsCopy.
 

OodleNetwork1TCP_State_InitAsCopyOodleAPI_OodleNetwork1OodleNetwork1TCP_Encode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
Navigation
 FAQ: Which OodleLZ should I use?
 Frequently Asked Questions
 FAQ: How do I decompress to graphics memory quickly?
 Welcome to Oodle
 Change Log

To test the various options for compressor & level, the best option is to run example_lz_chart : Example that makes a chart of OodleLZ options. This lets you generate numbers on your own machine, on your own file.

In general, OodleLZ_CompressionLevel always trades off worse encode speed for compression ratio. It's the amount of encoder effort, and typically doesn't affect decode speed much (though better compression also leads to better decode speed).

Here is a sample run of example_lz_chart showing the choice of OodleLZ_Compressor vs OodleLZ_CompressionLevel. See also FAQ: Which OodleLZ should I use? and About OodleLZ.

Oodle 2.6.3 example_lz_chart <file>
lz_chart loading r:testsetslztestsetlzt99...
file size : 24700820

Selkie : super fast to encode & decode, least compression Mermaid: fast decode with better-than-zlib compression Kraken : good compression, fast decoding, great tradeoff! Leviathan : very high compression, slowest decode
chart cell shows | raw/comp ratio : encode MB/s : decode MB/s | All compressors run at various encoder effort levels (SuperFast - Optimal). Many repetitions are run for accurate timing.
| HyperFast4| HyperFast3| HyperFast2| HyperFast1| SuperFast | Selkie |1.41:675:3895|1.45:622:3888|1.53:465:3696|1.68:369:3785|1.70:342:3759| Mermaid|1.66:436:2189|1.66:436:2188|1.79:352:2090|2.01:276:2055|2.04:261:2025| Kraken |1.55:588:1839|1.71:419:1136|1.88:331:1087|2.10:279:1093|2.27:167:1010|
compression ratio (raw/comp): | HyperFast4| HyperFast3| HyperFast2| HyperFast1| SuperFast | Selkie | 1.412 | 1.447 | 1.526 | 1.678 | 1.698 | Mermaid| 1.660 | 1.660 | 1.793 | 2.011 | 2.041 | Kraken | 1.548 | 1.711 | 1.877 | 2.103 | 2.268 |
encode speed (MB/s): | HyperFast4| HyperFast3| HyperFast2| HyperFast1| SuperFast | Selkie | 674.548 | 621.811 | 464.555 | 369.364 | 341.588 | Mermaid| 435.650 | 435.923 | 352.475 | 276.199 | 260.511 | Kraken | 588.488 | 418.921 | 331.423 | 279.129 | 167.206 |
decode speed (MB/s): | HyperFast4| HyperFast3| HyperFast2| HyperFast1| SuperFast | Selkie | 3894.644 | 3887.820 | 3695.984 | 3785.457 | 3758.594 | Mermaid| 2189.030 | 2187.863 | 2090.319 | 2054.897 | 2024.692 | Kraken | 1839.091 | 1135.920 | 1086.922 | 1093.407 | 1009.967 |
| VeryFast | Fast | Normal | Optimal1 | Optimal3 | Selkie |1.75:205:3490|1.83:105:3687|1.86: 43:3815|1.93:5.1:3858|1.94:2.6:3856| Mermaid|2.12:173:1991|2.19: 84:2177|2.21: 32:2291|2.37:2.8:2058|2.44:1.8:1978| Kraken |2.32:112:1104|2.39: 37:1187|2.43: 20:1189|2.55:3.1:1103|2.65:1.2:1038| Leviath|2.50: 31: 738|2.57: 17: 787|2.62:9.5: 807|2.71:1.6: 811|2.76:0.9: 776|
compression ratio (raw/comp): | VeryFast | Fast | Normal | Optimal1 | Optimal3 | Selkie | 1.748 | 1.833 | 1.863 | 1.933 | 1.943 | Mermaid| 2.118 | 2.194 | 2.207 | 2.367 | 2.437 | Kraken | 2.320 | 2.390 | 2.434 | 2.551 | 2.646 | Leviath| 2.504 | 2.572 | 2.617 | 2.707 | 2.756 |
encode speed (MB/s): | VeryFast | Fast | Normal | Optimal1 | Optimal3 | Selkie | 204.621 | 104.758 | 42.504 | 5.102 | 2.554 | Mermaid| 172.681 | 84.227 | 32.030 | 2.798 | 1.836 | Kraken | 111.858 | 37.126 | 19.859 | 3.091 | 1.204 | Leviath| 31.031 | 16.697 | 9.461 | 1.621 | 0.869 |
decode speed (MB/s): | VeryFast | Fast | Normal | Optimal1 | Optimal3 | Selkie | 3490.442 | 3686.689 | 3814.655 | 3857.857 | 3856.226 | Mermaid| 1991.442 | 2176.725 | 2291.498 | 2057.575 | 1977.721 | Kraken | 1104.172 | 1186.638 | 1189.372 | 1103.148 | 1038.352 | Leviath| 737.934 | 787.152 | 806.523 | 811.161 | 775.800 |

 
FAQ: Which OodleLZ should I use?Frequently Asked QuestionsFAQ: How do I decompress to graphics memory quickly?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_PackedRawOverlap
Navigation
 OodleLZ_Compressor
 OodleLZ_CheckCRC
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_PackedRawOverlap
{
    OodleLZ_PackedRawOverlap_No = 0,
    OodleLZ_PackedRawOverlap_Yes = 1,
    OodleLZ_PackedRawOverlap_Force32 = 0x40000000
};
Discussion
Bool enum
 
OodleLZ_CompressorOodleAPI_LZ_CompressorsOodleLZ_CheckCRC

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetSeekEntryPackedPos
Navigation
 OodleLZ_FindSeekEntry
 OodleLZ_CompressionLevel_GetName
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S64 OodleLZ_GetSeekEntryPackedPos( OO_S32 seekI,
                                      const OodleLZ_SeekTable * seekTable );
Discussion
Get the compressed position of a seek entry
Parameters
seekI  seek entry index , in [0,numSeekEntries)
seekTable  result of OodleLZ_CreateSeekTable
Return Value
return  compressed buffer position of the start of this seek entry
Discussion


 

OodleLZ_FindSeekEntryOodleAPI_LZ_CompressorsOodleLZ_CompressionLevel_GetName

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNet_Plugins_SetPrintf
Navigation
 OodleNet_Plugins_SetJobSystemAndCount
 OodleNet_Plugins_SetAssertion
 Welcome to Oodle
 Change Log
// Function prototype:
t_fp_OodleNet_Plugin_Printf * OodleNet_Plugins_SetPrintf( t_fp_OodleNet_Plugin_Printf * fp_rrRawPrintf );
Discussion
Install the callback used by Oodle Core for logging
Parameters
fp_rrRawPrintf  function pointer to your log function; may be NULL to disable all logging
Return Value
return  returns the previous function pointer
Discussion

Use this function to install your own printf for Oodle Core.

The default implementation in debug builds, if you install nothing, uses the C stdio printf for logging. On Microsoft platforms, it uses OutputDebugString and not stdio.

To disable all logging, call OodleNet_Plugins_SetPrintf(NULL)

WARNING : this function is NOT thread safe! It should be done only once and done in a place where the caller can guarantee thread safety.

In the debug build of Oodle, you can install OodleNet_Plugin_Printf_Verbose to get more verbose logging


 

OodleNet_Plugins_SetJobSystemAndCountNetwork pluginsOodleNet_Plugins_SetAssertion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_SetCallback
Navigation
 OodleXLog_GetEcho
 OodleXLog_GetCallback
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXLog_SetCallback( OodleXLogCallbackRet * cb );
Discussion
Set the OodleXLogCallbackRet used for logs
Parameters
cb  the callback to use
Discussion

The callback is only called if the OODLEXLOG_CALLBACK bit is set in OodleXLog_SetState
 

OodleXLog_GetEchoOodleX Debug aidsOodleXLog_GetCallback

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_CanDecodeInCircularWindow
Navigation
 OodleLZ_Compressor_UsesLargeWindow
 OodleLZ_Compressor_CanDecodeThreadPhased
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_CanDecodeInCircularWindow( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor can be decoded using a fixed size circular window. deprecated as of 2.9.0
 
OodleLZ_Compressor_UsesLargeWindowOodleAPI_LZ_CompressorsOodleLZ_Compressor_CanDecodeThreadPhased

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetAllChunksCompressor
Navigation
 OodleLZ_GetCompressedStepForRawStep
 OodleLZ_GetFirstChunkCompressor
 Welcome to Oodle
 Change Log
// Function prototype:
OodleLZ_Compressor OodleLZ_GetAllChunksCompressor( const void * compBuf,
                                                   OO_SINTa compBufSize,
                                                   OO_SINTa rawLen );
Discussion
ask who compressed all chunks in this buf chunk
Parameters
compBuf  pointer to compressed data; must be the start of compressed buffer, or a step of OODLELZ_BLOCK_LEN raw bytes
compBufSize  size of compBuf
rawLen  rawlen of data in compBuf
Return Value
return  the OodleLZ_Compressor used to encode this chunk
Discussion

returns a simple compressor (for example OodleLZ_Compressor_Kraken) if that was used on all chunks

returns OodleLZ_Compressor_Hydra if different NewLZ encoders were used (for example Kraken+Mermaid)

returns OodleLZ_Compressor_Count if a heterogenous mix of compressors was used (not just NewLZ)

returns OodleLZ_Compressor_Invalid on error

note this is only for this chunk - later chunks may have different compressors (eg. with Hydra) if you compressed all chunks the same it's up to you to store that info in your header

returns OodleLZ_Compressor_Invalid if compBufSize is too small or any chunk is corrupt
 

OodleLZ_GetCompressedStepForRawStepOodleAPI_LZ_CompressorsOodleLZ_GetFirstChunkCompressor

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_Free
Navigation
 t_fp_OodleNet_Plugin_MallocAligned
 t_fp_OodleNet_Plugin_RunJob
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleNet_Plugin_Free )( void * ptr );
Discussion
Function pointer type for OodleFree
Return Value
return  pointer to memory to free
Discussion


 

t_fp_OodleNet_Plugin_MallocAlignedNetwork pluginst_fp_OodleNet_Plugin_RunJob

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_MallocAligned
Navigation
 t_fp_OodleNet_Plugin_Free
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void *( OODLE_CALLBACK t_fp_OodleNet_Plugin_MallocAligned )( OO_SINTa bytes,
                  OO_S32 alignment );
Discussion
Function pointer type for OodleMallocAligned
Parameters
bytes  number of bytes to allocate
alignment  required alignment of returned pointer
Return Value
return  pointer to memory allocated (must not be NULL)
Discussion

alignment will always be a power of two

alignment will always be >= OODLE_MALLOC_MINIMUM_ALIGNMENT


 

OodleNet_Plugins_SetAssertionNetwork pluginst_fp_OodleNet_Plugin_Free

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_IOAlignUpS32
Navigation
 OodleX_MakeExtensionKey
 OodleX_IOAlignUpS64
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleX_IOAlignUpS32( const OO_S32 x );
Discussion
Align up to OODLEX_IO_MAX_ALIGNMENT
Parameters
x  value to align
Return Value
return  aligned value
Discussion

Align x up to OODLEX_IO_MAX_ALIGNMENT
 

OodleX_MakeExtensionKeyOodleX UtilsOodleX_IOAlignUpS64

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMallocBig
Navigation
 OodleXMallocBigAlignment
 OodleXFreeBig
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXMallocBig( OO_SINTa bytes );
Discussion
alloc a large block with "Big" alignment
Parameters
bytes  the amount to allocate (must be > 0)
Return Value
return  pointer to allocated memory
Discussion

query the alignment via OodleXMallocBigAlignment
 

OodleXMallocBigAlignmentOodleX Memory AllocatorsOodleXFreeBig

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_SetUsageWarnings
Navigation
 Oodle_SetConfigValues
 Oodle_CheckVersion
 Welcome to Oodle
 Change Log
// Function prototype:
void Oodle_SetUsageWarnings( Oodle_UsageWarnings state );
Discussion
Enables or disables Oodle usage warnings.
Parameters
state  whether usage warnings should be enabled or disabled.
Discussion

Usage warnings are enabled by default and try to be low-noise, but in case you want to disable them, this is how.

This should generally be done once at startup. Setting this state while there are Oodle calls running on other threads has undefined results.
 

Oodle_SetConfigValuesCore BaseOodle_CheckVersion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_Encode
Navigation
 OodleNetwork1UDP_State_Size
 OodleNetwork1UDP_Decode
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1UDP_Encode( const OodleNetwork1UDP_State * state,
                                  const OodleNetwork1_Shared * shared,
                                  const void * raw,
                                  OO_SINTa rawLen,
                                  void * comp );
Discussion
Encode a packet
Parameters
state  const shared compression state
shared  const shared compression context
raw  packet bytes to compress
rawLen  size of the packet to compress ; can be >= 0
comp  output compressed bytes; must be allocated to at least OodleNetwork1_CompressedBufferSizeNeeded bytes
Return Value
return  length of output compressed data written to comp ; the returned compLen is strictly <= rawLen
Discussion

Encodes one packet.

state and shared are both const and can be shared by all encoders and decoders.

The returned compLen will never be greater than rawLen, because OodleNetwork1 won't send packets that expand under compression (it just sends them uncompressed) - however it may write further than that during the compression attempt. Do not use the returned compLen to check the size of the compressed buffer needed.


 

OodleNetwork1UDP_State_SizeOodleAPI_OodleNetwork1OodleNetwork1UDP_Decode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle Network and Data SDK separation
Navigation
 Welcome to Oodle
 Change Log

As of Oodle 2.7.0 , Oodle Network is now a separate SDK from Oodle Data.

Oodle Data compression provides two libraries (Core and Ext). Oodle Network provides another library.

For example on Windows you have :

In the Data SDK :

oo2core_win64.lib oo2ext_win64.lib

In the Network SDK :

oo2net_win64.lib

Oodle Data & Network are independent and do not link together.

If you are using Network & Data of the same SDK version, you may install the SDKs to the same place, allow them to overwrite shared files that are identical in the two SDKs.

The Oodle Network examples build without Oodle Data, but can optionally be set to use Oodle Data to compress the shared model state.

Oodle Core and Oodle Network have similar but independent plugins that the client can set to control their interaction with the system via logging, allocations, and assertions. If you use both SDKs, you can install your functions by calling both plugins, for example OodleNet_Plugins_SetAllocators and OodleCore_Plugins_SetAllocators.

If you do Oodle Ext Init, the Oodle Ext plugins are installed to Oodle Core. Oodle Ext does not interact with Oodle Network.
 

   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXInitOptions
Navigation
 OodleXConfigValues
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleXInitOptions
{
    const OodleXMallocVTable * m_pBaseVTable;
    OO_BOOL m_OodleInit_DebugAllocator;
    OO_S32 m_num_handles_log2;
    OO_BOOL m_OodleInit_ThreadLog;
    OO_BOOL m_OodleInit_Log;
    OO_BOOL m_OodleInit_Log_Header;
    const char * m_OodleInit_Log_FileName;
    OO_BOOL m_OodleInit_Log_FlushEachWrite;
    OO_BOOL m_OodleInit_BreakOnLogError;
    OO_BOOL m_OodleInit_Telemetry;
    void * m_OodleInit_Telemetry_Context;
    OO_BOOL m_OodleInit_StackTrace;
    OO_BOOL m_OodleInit_LeakTrack;
    OO_BOOL m_OodleInit_SimpleProf;
    OO_BOOL m_OodleInit_FuzzTest;
    t_OodleFPVoidVoid * m_OodleInit_ThreadProfiler_funcptr;
    OO_BOOL m_OodleInit_IOQ;
    OO_BOOL m_OodleInit_IOQ_Log;
    OO_BOOL m_OodleInit_IOQ_BreakOnError;
    OO_BOOL m_OodleInit_IOQ_Threaded;
    OO_BOOL m_OodleInit_IOQ_CheckAlignment;
    OO_BOOL m_OodleInit_Workers;
    OO_S32 m_OodleInit_Workers_Count;
};
Discussion
Options struct for OodleX_Init can be filled with OodleX_Init_GetDefaults
Members
m_pBaseVTable  vtable for OodleMalloc to use [OodleXMalloc_GetVTable_OS]
m_OodleInit_DebugAllocator  option : put a debug allocator layer on top of m_pBaseVTable [false]
m_num_handles_log2  log2 of the number of handle 2048 item chunks for the OodleXHandleTable [13]
m_OodleInit_ThreadLog  option : enable the ThreadLog [true]
m_OodleInit_Log  option : enable the Log [true]
m_OodleInit_Log_Header  option : write a header to the Log at startup [true]
m_OodleInit_Log_FileName  set the log file name (NULL for default, which is described in About Oodle on Platforms)
m_OodleInit_Log_FlushEachWrite  option : flush the log after each write [false]
m_OodleInit_BreakOnLogError  option : debug break when Oodle logs an error
m_OodleInit_Telemetry  option : make the Telemetry connection for tracking Oodle [false]
m_OodleInit_Telemetry_Context  the telemetry context, NULL means I will make it (if m_OodleInit_Telemetry is true)
m_OodleInit_StackTrace  option : enable stack tracing in Oodle [true]
m_OodleInit_LeakTrack  option : enable LeakTrack in Oodle
m_OodleInit_SimpleProf  option : enable simple profiler (this is mainly for me)
m_OodleInit_FuzzTest  deprecated , does nothing
m_OodleInit_ThreadProfiler_funcptr  option : enable the thread profiler
m_OodleInit_IOQ  option : enable the IOQ
m_OodleInit_IOQ_Log  option : enable logging operations on the IOQ
m_OodleInit_IOQ_BreakOnError  option : make the IOQ issue a debug break on any error (for debugging)
m_OodleInit_IOQ_Threaded  option : enable threading on IOQ (turn off for debugging)
m_OodleInit_IOQ_CheckAlignment  option : should IOQ check alignment of parameters?
m_OodleInit_Workers  option : enable the worker thread system
m_OodleInit_Workers_Count  number of worker threads to start (default is OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES)

 
OodleX_Shutdown_DebugBreakOnLeaksOodleX Startup and ShutdownOodleXConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Wait
Navigation
 OodleX_GetStatus
 OodleX_WaitAll
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleX_Wait( OodleXHandle h,
                          OodleXHandleDeleteIfDone deleteIfDone OODEFAULT( OodleXHandleDeleteIfDone_No ) );
Discussion
Block the calling thread until handle is not Pending
Parameters
h  OodleXHandle weak reference
deleteIfDone  if OodleXHandleDeleteIfDone_Yes, handle will be deleted
Return Value
return  handle status
Discussion

Will not return OodleXStatus_Pending.

OodleX_WaitNoDelete and OodleX_WaitAndDelete are macros that are provided as short-hands for OodleX_Wait. They are :

  
  #define OodleX_WaitNoDelete (h)    OodleX_Wait(h,OodleXHandleDeleteIfDone_No)
  #define OodleX_WaitAndDelete(h)    OodleX_Wait(h,OodleXHandleDeleteIfDone_Yes)


 

OodleX_GetStatusOodleX async handle operationsOodleX_WaitAll

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_network_client : Example with simple network client support
Navigation
 example_lz_threadphased : Example of 2-thread ThreadPhased decoding
 Examples
 example_packet : Example demonstrating network packet compression
 Welcome to Oodle
 Change Log
Discussion
Oodle example_network_client

This is an example of an Oodle Network UDP client (such as in a game's runtime).

It loads a previously trained model, such as created by example_packet : Example demonstrating network packet compression. This would be done offline by your game's tools.

The runtime component loads a previously created model and uses it to compress packets on the fly.

This example uses only Oodle2 Network lib.

(Oodle2 data compression is optional)


#include "../include/oodle2net.h"

// optional, use Oodle LZ for the network model data
//#define USE_OODLE_LZ_DATA_COMPRESSION

#ifdef USE_OODLE_LZ_DATA_COMPRESSION
#include "../include/oodle2.h"
#endif

#include "ooex.h" // example helpers

#ifndef _CRT_SECURE_NO_WARNINGS
#define _CRT_SECURE_NO_WARNINGS
#endif
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <stdlib.h>

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_network_client
#endif

#include "read_whole_file.h"

example_packet_file should contain a capture of packets from a real game session. It should be at least 100 MB.

See Capturing Training data for OodleNetwork

Some portion (as set by example_packet_test_holdout_fraction_denominator) will be held out for testing the compression level. The remainder will be used for training. In real game usage, you don't need to hold out any for testing, this is just for evaluating the compression level. To be fair, the packets used in training are not used in testing.

The file format is :

packet.bin : OO_U32 [LE] : numbers of channels (num_channels) repeatedly : { OO_U32 [LE] : channel index in [0,num_channels-1] OO_U32 [LE] : number of bytes of data in this packet (num_bytes) OO_U8 * num_bytes : payload of this packet }


static const char * c_example_packet_file = "r:\\packet.bin"; // <- change this to your file name

// runtimedata file name written by "example_packet" :
static const char * c_runtimedata_fileName = "example_packet_on1udpnew_runtimedata.bin";

// OodleNetworkRuntimeData is held by the client & server for runtime encoding or decoding
struct OodleNetworkRuntimeData;

// load a previously saved network compression model :
static OodleNetworkRuntimeData * OodleNetwork_CreateFromFile( const char * fileName );

// release the compressor memory :
static void OodleNetwork_Destroy(OodleNetworkRuntimeData * pCompressor);

// OodleNetwork_Encode returns compressed size
static OO_SINTa OodleNetwork_Encode(
            const OodleNetworkRuntimeData * c,
            const void * raw, OO_SINTa rawLen,
            void * comp );

// OodleNetwork_Decode returned decompressed size, or 0 for error
static OO_SINTa OodleNetwork_Decode(
            const OodleNetworkRuntimeData * c,
        const void * comp,  OO_SINTa compLen,
        void * raw );

extern "C" int main(int argc,char *argv[])
{
    printf("example_network_client [packet_file] [runtimedata_file]\n");

    // defaults :
    const char * example_packet_file = c_example_packet_file;
    const char * runtimedata_file = c_runtimedata_fileName;
    
    // override from command line args :
    if ( argc >= 2 )
    {
        example_packet_file = argv[1];
        if ( argc >= 3 )
            runtimedata_file = argv[2];
    }

    
---------------------------------------------------------------------

Load the previously trained model data

This compressor is const and will be used to compress all packets

---------------------------------------------------


    printf("Loading runtimedata file : %s\n",runtimedata_file);

    OodleNetworkRuntimeData * c = OodleNetwork_CreateFromFile( runtimedata_file );
    
    if ( c == NULL )
    {
        printf("FAILED!\n");
        return 10;
    }
    
    
-----------------------------------------------------

Load some test packet data from a file

-----------------------------------

    
    printf("Loading packet file : %s\n",example_packet_file);
    
    void * packet_file_data;
    OO_SINTa packet_file_size;
    packet_file_data = read_whole_file(example_packet_file,&packet_file_size);
    if ( packet_file_data == NULL )
    {
        printf("ERROR : couldn't load packet file\n");
        OodleNetwork_Destroy(c);
        return 10;
    }
    
    
-----------------------------------------------------

Use the OodleNetwork model to encode & decode the packets :

--------------------------------------

    
    printf("Testing compression...\n"); 
    
    // point at the packet data :   
    const OO_U8 * packet_bin_buf = (const OO_U8 *) packet_file_data;
    OO_SINTa packet_bin_size = packet_file_size;
    
    // get the number of channels ; first dword in packet bin : 
    const OO_U8 * packet_bin_ptr = packet_bin_buf;
    const OO_U8 * packet_bin_end = packet_bin_ptr + packet_bin_size;
    OO_S32 num_channels = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
    // num_channels ignored for UDP
    OOEX_UNUSED_VARIABLE(num_channels);
    
    // max packet size is just used for the stack buffer sizes :
    #define MAX_PACKET_SIZE  16384
    // technically comp_buffer should be OodleNetwork1_CompressedBufferSizeNeeded(MAX_PACKET_SIZE) bytes
    
    OO_U8 comp_buffer[MAX_PACKET_SIZE];
    OO_U8 decomp_buffer[MAX_PACKET_SIZE];
    
    OO_S64 tot_rawLen = 0;
    OO_S64 tot_compLen = 0;
    OO_S64 tot_numPackets = 0;
        
    while( packet_bin_ptr < packet_bin_end )
    {
        // grab the current packet header :
        OO_S32 channel = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_UNUSED_VARIABLE(channel);
        OO_S32 bytes = OOEX_GET32_LE(packet_bin_ptr); packet_bin_ptr += sizeof(OO_S32);
        OOEX_ASSERT( channel >= 0 && channel < num_channels );
        OOEX_ASSERT( bytes > 0 );
        
        if ( packet_bin_ptr + bytes > packet_bin_end )
        {
            // partial end packet
            break;
        }
        
        OOEX_ASSERT( bytes < MAX_PACKET_SIZE );
        
        // compress one packet : (right before sending over the network)
        
        OO_SINTa cur_compLen = 
            OodleNetwork_Encode(c,packet_bin_ptr,bytes,comp_buffer);
        
        // [comp_buffer,cur_compLen] is sent over the network
        
        // check I'm not lying about compLen :
        comp_buffer[cur_compLen] = 0xCD;
        
        // decompress it : (upon receipt from the net)
        
        OO_SINTa decode_rawLen = OodleNetwork_Decode(c,
                    comp_buffer,cur_compLen,
                    decomp_buffer);
        OOEX_ASSERT_ALWAYS( decode_rawLen == bytes );

        // verify we got the packet back :
        OOEX_ASSERT_ALWAYS( memcmp(packet_bin_ptr,decomp_buffer,bytes) == 0 );
        
        packet_bin_ptr += bytes;            
        tot_rawLen += bytes;
        tot_compLen += cur_compLen;
        tot_numPackets ++;
    }
    
    free(packet_file_data);
    
    printf("%d packets compressed %.2f -> %.2f average\n",(int)tot_numPackets,
        (double)tot_rawLen/tot_numPackets,(double)tot_compLen/tot_numPackets);
    
    //-----------------------------------------------------
    
    // release the OodleNetwork memory :
    OodleNetwork_Destroy(c);

    return 0;
}

=================================================================================

Sample packet size transmission using "encodemod"

Packet size is sent in 1 byte if possible, else 2 bytes

This encoding assumes a max packet size is known.

c_packetsize_modbits should be set as low as possible, while still ensuring c_packetsize_max is large enough to send the largest packet size.

For example if your max packet size is 1500 , you could use modbits=3


static const int c_packetsize_modbits = 6;
static const int c_packetsize_immediate = 256 - (1<<c_packetsize_modbits); // = 192 for modbits=6
static const int c_packetsize_max = c_packetsize_immediate + (1<<(c_packetsize_modbits+8)); // =16576 for modbits = 6

static OO_U8 * PutPacketSize(OO_U8 * ptr,OO_SINTa size)
{
    OOEX_ASSERT( size < c_packetsize_max );
    if ( size < c_packetsize_immediate )
    {
        *ptr++ = (OO_U8) size;
    }
    else
    {
        size -= c_packetsize_immediate;
        OO_U8 lo = (OO_U8) size;
        OO_U8 hi = (OO_U8) (size>>8);
        OOEX_ASSERT( hi < 256-c_packetsize_immediate );
        *ptr++ = c_packetsize_immediate + hi;
        *ptr++ = lo;
    }
    return ptr;
}

static const OO_U8 * GetPacketSize(const OO_U8 * ptr,OO_SINTa * psize)
{
    OO_U8 first = *ptr++;
    if ( first < c_packetsize_immediate )
    {
        *psize = first;
    }
    else
    {
        int hi = first - c_packetsize_immediate;
        int lo = *ptr++;
        *psize = (hi<<8) + lo + c_packetsize_immediate;
    }
    return ptr;
}

//=================================================================================     

OodleNetwork1_SavedModel_Header is an example header for the saved Oodle Network trained model.

This is copied from example_packet

You could of course use your own header, and your own engine's IO code to persist this data.

      

struct OodleNetwork1_SavedModel_Header
{
    #define ON1_MAGIC   0x11235801
    OO_U32 magic;
    OO_U32 compressor;
    OO_U32 ht_bits;
    OO_U32 dic_size;
    OO_U32 oodle_major_version;
    OO_U32 dic_complen;
    OO_U32 statecompacted_size;
    OO_U32 statecompacted_complen;
};

OodleNetwork1_SavedModel_Header is written to file like a flat struct, but we ensure it's always little endian.

static void OodleNetwork1_SavedModel_Header_Read(OodleNetwork1_SavedModel_Header * pHeader,const void * from_memory)
{
    const OO_U32 * from_ptr = (const OO_U32 *)from_memory;
    pHeader->magic = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->compressor = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->ht_bits = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->dic_size = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->oodle_major_version = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->dic_complen = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->statecompacted_size = OOEX_GET32_LE(from_ptr); from_ptr++;
    pHeader->statecompacted_complen = OOEX_GET32_LE(from_ptr); from_ptr++;
    OOEX_ASSERT( (((OO_U8 *)from_ptr) - ((OO_U8 *)from_memory)) == sizeof(OodleNetwork1_SavedModel_Header) );
}

struct OodleNetworkRuntimeData
{
    void * dic;
    OodleNetwork1UDP_State * state;
    OodleNetwork1_Shared * shared;
};


static OO_SINTa OodleNetwork_Encode(
            const OodleNetworkRuntimeData * c,
            const void * raw, OO_SINTa rawLen,
            void * comp )
{
    // put uncompressed packet size first :
    // NOTE : if you have this in some kind of header already, then don't put it again here!
    OO_U8 * compPtr = (OO_U8 *)comp;
    compPtr = PutPacketSize(compPtr,rawLen);
    // then put the compressed packet :
    compPtr += OodleNetwork1UDP_Encode(c->state,c->shared,raw,rawLen,compPtr);
    return compPtr - (OO_U8 *)comp;
}


static OO_SINTa OodleNetwork_Decode(
            const OodleNetworkRuntimeData * c,
        const void * comp,  OO_SINTa compLen,
        void * raw )
{       
    const OO_U8 * compPtr = (const OO_U8 * )comp;
    OO_SINTa rawLen;
    // first get the uncompressed packet size :
    compPtr = GetPacketSize(compPtr,&rawLen);
    OO_SINTa packet_compLen = ((const OO_U8 * )comp + compLen) - compPtr;
    // try to check that the packet size we got makes sense :
    // it should be >= compressed size because OodleNetwork never expands :
    OOEX_ASSERT( rawLen >= packet_compLen );
    if ( rawLen < packet_compLen )
        return 0; // corruption!
    // then decode the packet :
    OO_BOOL ok = OodleNetwork1UDP_Decode(c->state,c->shared,compPtr,packet_compLen,
                    raw,rawLen);
    if ( ! ok )
        return 0;
    return rawLen;
}                   

static void OodleNetwork_Destroy(OodleNetworkRuntimeData * pCompressor)
{
    free(pCompressor->dic);
    free(pCompressor->shared);
    free(pCompressor->state);
    free(pCompressor);
}
    
static bool OodleNetwork_LoadFromFileData( OodleNetworkRuntimeData * pCompressor, const void * fileData, OO_SINTa fileSize )        
{
    // parse header :
    OodleNetwork1_SavedModel_Header header;
    OodleNetwork1_SavedModel_Header_Read(&header,fileData);
    
    if ( header.magic != ON1_MAGIC )
    {
        printf("ERROR : state_file ON1_MAGIC mismatch\n");
        return false;
    }
    
    #if 0
    // optional check for oodle_major_version being the same
    // NOTE : this is more conservative than necessary
    //  you may disable this check to keep loading old state files
    OO_U32 oodle_major_version_major = header.oodle_major_version;
    if ( oodle_major_version_major < 1 )
    {
        OodleXLog_Printf_v0("state_file version too old!\n");
        return false;
    }
    if ( oodle_major_version_major > OODLE2NET_VERSION_MAJOR )
    {
        OodleXLog_Printf_v0("state_file version newer that SDK!\n");
        return false;
    }
    #endif
    
    OO_S32 on1udpnew_ht_bits = header.ht_bits;
    OO_SINTa on1udpnew_dic_size = header.dic_size;
    
    OO_SINTa on1udpnew_dic_complen = header.dic_complen;
    OO_SINTa on1udpnew_statecompacted_size = header.statecompacted_size;
    OO_SINTa on1udpnew_statecompacted_complen = header.statecompacted_complen;
    
    OOEX_ASSERT_ALWAYS( on1udpnew_dic_size >= on1udpnew_dic_complen );
    OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size >= on1udpnew_statecompacted_complen );
    
    OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size > 0 && on1udpnew_statecompacted_size < OodleNetwork1UDP_StateCompacted_MaxSize() );
    
    OOEX_ASSERT_ALWAYS( fileSize == (OO_S64)sizeof(OodleNetwork1_SavedModel_Header) + on1udpnew_dic_complen + on1udpnew_statecompacted_complen );
    
//  printf("OodleNetwork1UDP Loading; dic comp %d , state %d->%d\n",
//      (int)on1udpnew_dic_complen,(int)on1udpnew_statecompacted_size,(int)on1udpnew_statecompacted_complen);
    
    //-------------------------------------------
    // decompress on1udpnew_dic and on1udpnew_statecompacted

    pCompressor->dic = malloc(on1udpnew_dic_size);

    const void * on1udpnew_dic_comp_ptr = (const OO_U8 *)(fileData) + sizeof(OodleNetwork1_SavedModel_Header);

    OodleNetwork1UDP_StateCompacted * on1udpnew_compacted = (OodleNetwork1UDP_StateCompacted *) malloc( on1udpnew_statecompacted_size );

    void * on1udpnew_statecompacted_comp_ptr = (OO_U8 *)on1udpnew_dic_comp_ptr + on1udpnew_dic_complen;


    #ifdef USE_OODLE_LZ_DATA_COMPRESSION

    // the dictionary and the state might be compressed with Oodle LZ
    //  decode them :

    if ( header.compressor == (OO_U32)OodleLZ_Compressor_Invalid )
    {
        OOEX_ASSERT_ALWAYS( on1udpnew_dic_complen == on1udpnew_dic_size );

        memcpy(pCompressor->dic,on1udpnew_dic_comp_ptr,on1udpnew_dic_size);     
        
        OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size == on1udpnew_statecompacted_complen );

        memcpy(on1udpnew_compacted,on1udpnew_statecompacted_comp_ptr,on1udpnew_statecompacted_size);
    }
    else
    {
        OO_SINTa decomp_dic_size = OodleLZ_Decompress(on1udpnew_dic_comp_ptr,on1udpnew_dic_complen,pCompressor->dic,on1udpnew_dic_size,OodleLZ_FuzzSafe_Yes);
    
        OOEX_ASSERT_ALWAYS( decomp_dic_size == on1udpnew_dic_size );
        
        OO_SINTa decomp_statecompacted_size = OodleLZ_Decompress(on1udpnew_statecompacted_comp_ptr,on1udpnew_statecompacted_complen,on1udpnew_compacted,on1udpnew_statecompacted_size,OodleLZ_FuzzSafe_Yes);

        OOEX_ASSERT_ALWAYS( decomp_statecompacted_size == on1udpnew_statecompacted_size );
    }
    
    #else
    
    OOEX_ASSERT_ALWAYS( header.compressor == (OO_U32)-1 );
    
    OOEX_ASSERT_ALWAYS( on1udpnew_dic_complen == on1udpnew_dic_size );
    memcpy(pCompressor->dic,on1udpnew_dic_comp_ptr,on1udpnew_dic_size);
    
    OOEX_ASSERT_ALWAYS( on1udpnew_statecompacted_size == on1udpnew_statecompacted_complen );

    memcpy(on1udpnew_compacted,on1udpnew_statecompacted_comp_ptr,on1udpnew_statecompacted_size);
        
    #endif
        
    //OodleXFree_IOAligned(state_fileData); state_fileData = NULL;

    //----------------------------------------------
    // Uncompact the "Compacted" state into a usable state

    OO_SINTa on1udpnew_state_size = OodleNetwork1UDP_State_Size();
    pCompressor->state = (OodleNetwork1UDP_State *) malloc( on1udpnew_state_size );
    OOEX_ASSERT_ALWAYS( pCompressor->state != NULL );

    if ( ! OodleNetwork1UDP_State_Uncompact(pCompressor->state,on1udpnew_compacted) )
    {
        printf("OodleNetwork1UDP_State_Uncompact failed\n");
        return false;
    }
    free(on1udpnew_compacted);
    
    //----------------------------------------------
    // fill out on1udpnew_shared from the dictionary
            
    OO_SINTa shared_size = OodleNetwork1_Shared_Size(header.ht_bits);
    pCompressor->shared = (OodleNetwork1_Shared *) malloc( shared_size );
    OOEX_ASSERT_ALWAYS( pCompressor->shared != NULL );
    OodleNetwork1_Shared_SetWindow(pCompressor->shared,on1udpnew_ht_bits,pCompressor->dic,(OO_S32)on1udpnew_dic_size);
    
    return true;
}

static OodleNetworkRuntimeData * OodleNetwork_CreateFromFile( const char * fileName )
{
    void * fileData;
    OO_SINTa fileSize;
    fileData = read_whole_file(fileName,&fileSize);

    if ( ! fileData )
    {
        printf("ERROR : couldn't read file %s\n",fileName);
        return NULL;
    }

    OodleNetworkRuntimeData * c = (OodleNetworkRuntimeData *) malloc( sizeof(OodleNetworkRuntimeData) );
    memset(c,0,sizeof(*c));
    
    bool ok = OodleNetwork_LoadFromFileData(c,fileData,fileSize);

    free(fileData);

    if ( ! ok )
    {
        printf("ERROR : couldn't parse file %s\n",fileName);
        OodleNetwork_Destroy(c);
        return NULL;
    }
    
    return c;
}

//==================================================================================

 
example_lz_threadphased : Example of 2-thread ThreadPhased decodingExamplesexample_packet : Example demonstrating network packet compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleCore_Plugins_SetJobSystemAndCount
Navigation
 OodleCore_Plugins_SetJobSystem
 OodleCore_Plugins_SetPrintf
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleCore_Plugins_SetJobSystemAndCount( t_fp_OodleCore_Plugin_RunJob * fp_RunJob,
                                             t_fp_OodleCore_Plugin_WaitJob * fp_WaitJob,
                                             int target_parallelism );
Discussion
Set the function pointers for async job system needed by Oodle2 Core
Parameters
fp_RunJob  pointer to RunJob function
fp_WaitJob  pointer to WaitJob function
target_parallelism  goal of number of jobs to run simultaneously
Discussion

If these are not set, the default implementation runs jobs synchronously on the calling thread.

These must not be changed once they are set! Set them once then don't change them.

target_parallelism allows you to tell Oodle how many Jobs it should try to keep in flight at once. Depending on the operation it may not be able to split work into this many jobs (so fewer will be used), but it will not exceed this count.

For Oodle Data LZ work, typically target_parallelism is usually best at the number of hardware cores not including hyper threads).

For Oodle Texture BCN encoding work, target_parallelism is usually best as the full number of hyper cores.

In some cases you may wish to reduce target_parallelism by 1 or 2 cores to leave some of the CPU free for other work.

For example on a CPU with 16 cores and 32 hardware threads, for LZ work you might set target_parallelism to 15 when calling OodleCorePlugins. For BC7 encoding you might set target_parallelism to 30 when calling OodleTexPlugins.

NOTE : if you are using Oodle Ext, do NOT call this. OodleX_Init will install a job system for Oodle Core. Note OodleX only installs automatically to Oodle Core, not Net or Tex. See example_jobify.cpp for manual plugin.

Replaces deprecated OodleCore_Plugins_SetJobSystem

See About Oodle Job Threading Plugins
 

OodleCore_Plugins_SetJobSystemCore pluginsOodleCore_Plugins_SetPrintf

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleCore_Plugins_SetAllocators
Navigation
 OodleCore_Plugins_SetJobSystem
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleCore_Plugins_SetAllocators( t_fp_OodleCore_Plugin_MallocAligned * fp_OodleMallocAligned,
                                      t_fp_OodleCore_Plugin_Free * fp_OodleFree );
Discussion
Set the function pointers for allocation needed by Oodle2 Core
Discussion
If these are not set, the default implementation on most platforms uses the C stdlib. On Microsoft platforms the default implementation uses HeapAlloc.

These must not be changed once they are set! Set them once then don't change them.

NOTE: if you are using Oodle Ext, do NOT call this. OodleX_Init will install an allocator for Oodle Core. Do not mix your own allocator with the OodleX allocator. See OodleX Memory Allocators.

If you want to ensure that Oodle is not doing any allocations, you can call OodleCore_Plugins_SetAllocators(NULL,NULL); If you do that, then any time Oodle needs to allocate memory internally, it will stop the process. It is STRONGLY not recommended that you ship that way. You can verify that Oodle is not allocating, but then leave some fallback allocator installed when you actually ship just in case.

Also note that on many consoles the standard allocation practices may not leave much heap memory for the C stdlib malloc. In this case Oodle may fail to allocate.


 

Core pluginsCore pluginsOodleCore_Plugins_SetJobSystem

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetChunkCompressor
Navigation
 OodleLZ_GetFirstChunkCompressor
 Welcome to Oodle
 Change Log
// Function prototype:
OodleLZ_Compressor OodleLZ_GetChunkCompressor( const void * compChunkPtr,
                                               OO_SINTa compBufAvail,
                                               OO_BOOL * pIndependent );
Discussion
Deprecated entry point for backwards compatibility
Discussion
Use OodleLZ_GetFirstChunkCompressor or OodleLZ_GetAllChunksCompressor


 

OodleLZ_GetFirstChunkCompressorOodleAPI_LZ_CompressorsOodleDecompressCallback

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_FreeSeekTable
Navigation
 OodleLZ_CreateSeekTable
 OodleLZ_CheckSeekTableCRCs
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleLZ_FreeSeekTable( OodleLZ_SeekTable * pTable );
Discussion
Frees a table allocated by OodleLZ_CreateSeekTable
 
OodleLZ_CreateSeekTableOodleAPI_LZ_CompressorsOodleLZ_CheckSeekTableCRCs

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_DisplayAssertion
Navigation
 t_fp_OodleCore_Plugin_Printf
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC OO_BOOL( OODLE_CALLBACK t_fp_OodleCore_Plugin_DisplayAssertion )( const char * file,
                   const int line,
                   const char * function,
                   const char * message );
Discussion
Function pointer to Oodle Core assert callback
Parameters
file  C file that triggered the assert
line  C line that triggered the assert
function  C function that triggered the assert (may be NULL)
message  assert message
Return Value
return  true to break execution at the assertion site, false to continue
Discussion

This callback is called by Oodle Core when it detects an assertion condition.

This will only happen in debug builds.


 

t_fp_OodleCore_Plugin_PrintfCore pluginsCore LZ compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_IO_MAX_ALIGNMENT
Navigation
 OODLEX_BUFFER_SIZE_DEFAULT
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_IO_MAX_ALIGNMENT (4096)
Discussion
Oodle low level offsets and sizes are aligned to OODLEX_IO_MAX_ALIGNMENT
Discussion
Unbuffered IO (as in OodleX low level async io) require alignment to OODLEX_IO_MAX_ALIGNMENT. Pointers returned by OodleXMalloc_IOAligned are so aligned. You can also use the utility functions such as OodleX_IOAlignUpS32 to align values.
 
About OodleIOQOodleX low level async ioOODLEX_BUFFER_SIZE_DEFAULT

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_DecodeSome
Navigation
 OodleLZDecoder_Reset
 OodleLZDecoder_MakeValidCircularWindowSize
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZDecoder_DecodeSome( OodleLZDecoder * decoder,
                                   OodleLZ_DecodeSome_Out * out,
                                   void * decBuf,
                                   OO_SINTa decBufPos,
                                   OO_SINTa decBufferSize,
                                   OO_SINTa decBufAvail,
                                   const void * compPtr,
                                   OO_SINTa compAvail,
                                   OodleLZ_FuzzSafe fuzzSafe OODEFAULT( OodleLZ_FuzzSafe_No ),
                                   OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                                   OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                                   OodleLZ_Decode_ThreadPhase threadPhase OODEFAULT( OodleLZ_Decode_Unthreaded ) );
Discussion
Incremental decode some LZ compressed data
Parameters
decoder  the OodleLZDecoder, made by OodleLZDecoder_Create
out  filled with results
decBuf  the decode buffer (window)
decBufPos  the current position in the buffer
decBufferSize  size of decBuf ; this must be either equal to the total decompressed size (rawLen passed to OodleLZDecoder_Create) or the result of OodleLZDecoder_MakeValidCircularWindowSize
decBufAvail  the number of bytes available after decBufPos in decBuf ; usually (decBufferSize - decBufPos), but can be less
compPtr  pointer to compressed data to read
compAvail  number of compressed bytes available at compPtr
fuzzSafe  (optional) should the decode be fuzz safe
checkCRC  (optional) if data could be corrupted and you want to know about it, pass OodleLZ_CheckCRC_Yes
verbosity  (optional) if not OodleLZ_Verbosity_None, logs some info
threadPhase  (optional) for threaded decode; see About OodleLZ ThreadPhased Decode (default OodleLZ_Decode_Unthreaded)
Return Value
return  true if success, false if invalid arguments or data is encountered
Discussion

Decodes data encoded with an OodleLZ compressor.

Decodes an integer number of quanta; quanta are OODLELZ_QUANTUM_LEN uncompressed bytes.

decBuf can either be a circular window or the whole rawLen array. In either case, decBufPos should be in the range [0,decBufferSize). If decBuf is a circular window, then decBufferSize should come from OodleLZDecoder_MakeValidCircularWindowSize.

(circular windows are deprecated as of 2.9.0)

NOTE : all the new LZ codecs (Kraken, etc.) do not do circular windows. They can do sliding windows, see lz_test_11 in example_lz : Example demonstrating LZ compression and decompression. They should always have decBufferSize = total raw size, even if the decode buffer is smaller than that.

NOTE : insufficient data provided (with compAvail > 0 but not enough to decode a quantum) is a success case (return value of true), even though nothing is decoded. A return of false always indicates a non-recoverable error.

If decBufAvail or compAvail is insufficient for any decompression, the "curQuantum" fields of OodleLZ_DecodeSome_Out will tell you how much you must provide to proceed. That is, if enough compressed bytes are provided to get a quantum header, but not enough to decode a quantum, this function returns true and fills out the OodleLZ_DecodeSome_Out structure with the size of the quantum.

See OodleLZ_Decompress about fuzz safety.

NOTE : DecodeSome expect to decode either one full quantum (of len OODLELZ_QUANTUM_LEN) or up to the length of the total buffer specified in the call to OodleLZDecoder_Create or OodleLZDecoder_Reset. That total buffer length must match what was use during compression (or be a seek-chunk portion thereof). That is, you cannot decompress partial streams in intervals smaller than OODLELZ_QUANTUM_LEN except for the final partial quantum at the end of the stream.


 

OodleLZDecoder_ResetOodleAPI_LZ_CompressorsOodleLZDecoder_MakeValidCircularWindowSize

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State_Uncompact
Navigation
 OodleNetwork1UDP_State_Uncompact_ForVersion
 OodleNetwork1_SelectDictionarySupported
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1UDP_State_Uncompact( OodleNetwork1UDP_State * to,
                                          const OodleNetwork1UDP_StateCompacted * from );
Discussion
Fills a OodleNetwork1UDP_State from a OodleNetwork1UDP_StateCompacted
Parameters
to  filled out
from  read
Return Value
return  false if invalid data is detected
Discussion

Use this when the OodleNetwork1UDP_StateCompacted is read from a file, so that the OodleNetwork1UDP_State can be used for coding. You may discard the OodleNetwork1UDP_StateCompacted after this call.

to should be allocated to OodleNetwork1UDP_State_Size bytes.

NOTE : the return value here is not a robust way to detect tampered data. You should use encryption and/or a safe checksum on your saved model file data.


 

OodleNetwork1UDP_State_Uncompact_ForVersionOodleAPI_OodleNetwork1OodleNetwork1_SelectDictionarySupported

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_Train
Navigation
 OodleNetwork1TCP_Decode
 OodleNetwork1UDP_State_Size
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNetwork1UDP_Train( OodleNetwork1UDP_State * state,
                             const OodleNetwork1_Shared * shared,
                             const void * * training_packet_pointers,
                             const OO_S32 * training_packet_sizes,
                             OO_S32 num_training_packets );
Discussion
Fill a OodleNetwork1UDP_State from training data
Parameters
state  the OodleNetwork1UDP_State which is filled out; this state should not need to be initialized in any way before calling Train, it will be reset internally.
shared  the OodleNetwork1_Shared data to use in compression ; this shared data should already have had OodleNetwork1_Shared_SetWindow done on it.
training_packet_pointers  array of pointers to packet data; array of size num_training_packets
training_packet_sizes  array of sizes of packets; array of size num_training_packets
num_training_packets  number of packets
Discussion

OodleNetwork1UDP_Train uses the provided training packet data to initialize state.

The training packet data provided here should not overlap the window passed to OodleNetwork1_Shared_SetWindow ; it should not come from the same source or you will get false training.

You may call OodleNetwork1_Shared_SetWindow and OodleNetwork1UDP_Train many times with different windows to optimize the window selection.

Once training is done, the resulting OodleNetwork1UDP_State should be written to disk and used by both the client and server.

There's no need to copy a OodleNetwork1UDP_State , the same state object can be used by all encoders and decoders.
 

OodleNetwork1TCP_DecodeOodleAPI_OodleNetwork1OodleNetwork1UDP_State_Size

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXCopyFileFlags
Navigation
 OodleFileNotFoundIsAnError
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXCopyFileFlags
{
    OodleXCopyFileFlags_Overwrite = 0,
    OodleXCopyFileFlags_DontOverwriteExisting = 1,
    OodleXCopyFileFlags_OverwriteOnlyIfNewer = 2,
    OodleXCopyFileFlags_OverwriteOnlyIfDifferentSize = 4,
    OodleXCopyFileFlags_OverwriteOnlyIfNewerOrDifferentSize = 2|4,
    OodleXCopyFileFlags_Mask = 7,
    OodleXCopyFileFlags_Default = 0,
    OodleXCopyFileFlags_Force32 = 0x40000000
};
Discussion
Flags for Oodle CopyFile operations.
Enumerants
OodleXCopyFileFlags_Overwrite  always overwrite existing
OodleXCopyFileFlags_DontOverwriteExisting  never overwrite existing
OodleXCopyFileFlags_OverwriteOnlyIfNewer  overwrite only if source modtime is >= dest modtime
OodleXCopyFileFlags_OverwriteOnlyIfDifferentSize  overwrite if source size != dest size
OodleXCopyFileFlags_OverwriteOnlyIfNewerOrDifferentSize  common combo of overwrite options
OodleXCopyFileFlags_Mask 
OodleXCopyFileFlags_Default  default action Overwrite
OodleXCopyFileFlags_Force32 
Discussion
Combine with logical OR.


 

OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZEOodleX low level async ioOodleFileNotFoundIsAnError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_InstallVTable
Navigation
 OodleXMalloc_SetFailedHandler
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXMalloc_InstallVTable( const OodleXMallocVTable * pvt,
                                 const OodleXMallocVTable * pBaseVT );
Discussion
Install the vtable that OodleX will use to allocate memory
Parameters
pvt  pointer to the vtable to be installed (will be copied)
pBaseVT  if pvt is a layered vtable, this is the underlying alloc; if not it should be = pvt
Discussion

Sets the global vtable that will de used by the OodleXMalloc calls. Typically let OodleX_Init install a suitable vtable for you. If you do it manually, it must be done before any other OodleX initialization.

WARNING : You must not change the vtable after OodleX is running; pointers allocated from the previous vtable will still need to be freed and will call to the global vtable.
 

OodleXMallocVTableOodleX Memory AllocatorsOodleXMalloc_SetFailedHandler

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Semaphore_Post
Navigation
 OodleX_Semaphore_Wait
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_Semaphore_Post( OodleX_Semaphore * sem,
                            OO_S32 count OODEFAULT( 1 ) );
Discussion
OodleX_Semaphore_Post
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX threading utilOodleX threading utilOodleX_Semaphore_Wait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleCountdown_Decrement
Navigation
 OodleXHandleCountdown_Alloc
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXHandleCountdown_Decrement( OodleXHandle h,
                                              OO_S32 decCount OODEFAULT( 1 ) );
Discussion
Decrement a countdown handle created by OodleXHandleCountdown_Alloc
Parameters
h  handle allocated by OodleXHandleCountdown_Alloc
decCount  how much to decrement the countdown
Return Value
return  status after the decrement
Discussion

Returns OodleXStatus_Done if this decrement took the countdown to 0, else OodleXStatus_Pending.
 

OodleXHandleCountdown_AllocOodleX async handle operationsOodleXHandle

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_MemorySizeNeeded
Navigation
 OodleLZDecoder_Create
 OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZDecoder_MemorySizeNeeded( OodleLZ_Compressor compressor OODEFAULT( OodleLZ_Compressor_Invalid ),
                                        OO_SINTa rawLen OODEFAULT( - 1 ) );
Discussion
If you want to provide the memory needed by OodleLZDecoder_Create , this tells you how big it must be.
Parameters
compressor  the type of data you will decode; use OodleLZ_Compressor_Invalid if unknown
rawLen  should almost always be -1, which supports any size of raw data decompression
Return Value
return  bytes to allocate or reserve, 0 for failure
Discussion

NOTE : using OodleLZ_Compressor_Invalid lets you decode any time of compressed data. It requests as much memory as the largest compressor. This may be a lot more than your data needs; try to use the correct compressor type.

If rawLen is -1 (default) then the Decoder object created can be used on any length of raw data decompression. If rawLen is specified here, then you can only use it to decode data shorter than the length you specified here. This use case is very rare, contact support for details.
 

OodleLZDecoder_CreateOodleAPI_LZ_CompressorsOodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: How do I get the Oodle logs?
Navigation
 FAQ: How do I decompress to graphics memory quickly?
 Frequently Asked Questions
 FAQ: I write a file with IOQ but the contents are garbage?
 Welcome to Oodle
 Change Log

When you have a problem with Oodle, I might ask you to send me your log file. This is how to find it.

(if you are using Oodle Core only, with no Oodle X, Oodle does not write its own log file, that's up to you. You could link in OodleX and use it to enable logging. This gets OodleX to call OodlePlugins_SetPrintf to install its logger in Oodle Core. Another option with Oodle Core is to use OodlePlugins_SetPrintf with OodlePlugin_Printf_Verbose to get more detailed logging, but only in the debug version of Oodle).

NOTE : if you are using Core only on a Microsoft target, the default core log function only goes to OutputDebugString - not stdio - so you may not see any error messages unless you are running in the debugger. Either use Oodle X or install your own logger to get messages however you want.

If you are running Oodle Ext with the default options to OodleX_Init, a log file is written by default. If you are setting the options by hand, ensure OodleXInitOptions:m_OodleInit_Log is on. You can also toggle logging on the fly with OodleXLog_SetState.

To get more information in the log, call OodleXLog_SetVerboseLevel with OodleXLog_Verbose_Lots. You can also pass a higher level of OodleLZ_Verbosity to the OodleLZ_Decompress call.

If you know which function is failing, it helps to call OodleXLog_Printf with the value of the arguments that you are passing (Oodle does not automatically log the value of every argument passed).

If your app is crashing because of the error and you can't debug, you can set OodleXInitOptions:m_OodleInit_Log_FlushEachWrite ; hopefully this will get the log to flush out some information about the error before the crash.

You can set the log file name in OodleXInitOptions:m_OodleInit_Log_FileName. If you don't set it, Oodle will use a default name that depends on the platform (see About Oodle on Platforms for per-platform log info).

It may be useful to call OodleX_LogSystemInfo at startup (after calling OodleX_Init). This will write a bunch of information about your machine and your Oodle build version to the log file. It is not done automatically.

Please grab the entire log file and mail it to me, don't send just a portion of it.
 

FAQ: How do I decompress to graphics memory quickly?Frequently Asked QuestionsFAQ: I write a file with IOQ but the contents are garbage?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_Decode
Navigation
 OodleNetwork1UDP_Encode
 OodleNetwork1UDP_StateCompacted_MaxSize
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleNetwork1UDP_Decode( const OodleNetwork1UDP_State * state,
                                 const OodleNetwork1_Shared * shared,
                                 const void * comp,
                                 OO_SINTa compLen,
                                 void * raw,
                                 OO_SINTa rawLen );
Discussion
Decode a packet
Parameters
state  const shared compression state
shared  const shared compression context
comp  compressed packet received
compLen  size of compressed data
raw  output decompressed packet
rawLen  size of the packet to write
Return Value
return  false for failure
Discussion

Decodes one packet.

state and shared are both const and can be shared by all encoders and decoders.

The rawLen provided here must match the length used in OodleNetwork1UDP_Encode when creating this compressed packet. The OodleNetwork1 data is headerless, it's up to you to send the packet decompressed size in your own header.

If corrupt data is detected, false is returned.

If the number of compressed bytes consumed does not match compLen, false is returned. If the number of output bytes does not match rawLen, false is returned.

This function, however, does not do verify data integrity. It will return 'true' if the correct number of bytes are coded, even if the data does not match.

The buffer comp must be allowed to read at least compLen + OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN bytes (note that this is strictly less than OodleNetwork1_CompressedBufferSizeNeeded(rawLen) , so that may be used as well)


 

OodleNetwork1UDP_EncodeOodleAPI_OodleNetwork1OodleNetwork1UDP_StateCompacted_MaxSize

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQFile
Navigation
 OodleXOSFile
 Welcome to Oodle
 Change Log
// Type definition:
OO_U64 OodleXIOQFile
Discussion
Opaque weak reference handle to an IOQ File
 
OodleX_SetDefaultFileOpsOodleX low level async ioOodleXOSFile

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded
Navigation
 OodleLZDecoder_MemorySizeNeeded
 OodleLZDecoder_Destroy
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZ_ThreadPhased_BlockDecoderMemorySizeNeeded( void );
Discussion
Returns the size of the decoder needed for ThreadPhased decode
Discussion
For use with OodleLZ_Decode_ThreadPhase See About OodleLZ ThreadPhased Decode
 
OodleLZDecoder_MemorySizeNeededOodleAPI_LZ_CompressorsOodleLZDecoder_Destroy

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_SetInfoByName_AsyncAndWait
Navigation
 OodleXIOQ_GetInfoByName_AsyncAndWait
 OodleXIOQ_MakeAllDirs_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_SetInfoByName_AsyncAndWait( const char * name,
                                              OO_U32 flags,
                                              OO_U64 modTime );
Discussion
See OodleXIOQ_SetInfoByName_Async
 
OodleXIOQ_GetInfoByName_AsyncAndWaitOodleX low level async ioOodleXIOQ_MakeAllDirs_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetNumWorkerThreads
Navigation
 OodleX_CorePlugin_WaitJob
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleX_GetNumWorkerThreads( );
Discussion
Returns the number of worker threads
Discussion
When there are 0 worker threads, the OodleWork system still succeeds, it just runs Worklets synchronously on the calling thread.

The worker thread count is set in OodleXInitOptions:m_OodleInit_Workers_Count
 

OodleX_CorePlugin_WaitJobOodleX threading utilOodleX_Semaphore

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_GetName
Navigation
 OodleLZ_CompressionLevel_GetName
 OodleLZ_Jobify_GetName
 Welcome to Oodle
 Change Log
// Function prototype:
const char * OodleLZ_Compressor_GetName( OodleLZ_Compressor compressor );
Discussion
Provides a string naming a OodleLZ_Compressor compressor
 
OodleLZ_CompressionLevel_GetNameOodleAPI_LZ_CompressorsOodleLZ_Jobify_GetName

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetSeconds
Navigation
 OodleX_S64_to_SINTa_check
 OodleXUtil_ConvertUTF8ToUTF16
 Welcome to Oodle
 Change Log
// Function prototype:
OO_F64 OodleX_GetSeconds( );
Discussion
Get current time in seconds
Return Value
return  current time in seconds
Discussion

What it do.
 

OodleX_S64_to_SINTa_checkOodleX UtilsOodleXUtil_ConvertUTF8ToUTF16

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_CanDecodeInPlace
Navigation
 OodleLZ_Compressor_CanDecodeThreadPhased
 OodleLZ_Compressor_MustDecodeWithoutResets
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_CanDecodeInPlace( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor can be used with "in-place" decoding.

This is now always true (all compressors support in-place decoding). The function is left for backward compatibility.

All compressors in the future will support in-place, you don't need to check this property.


 

OodleLZ_Compressor_CanDecodeThreadPhasedOodleAPI_LZ_CompressorsOodleLZ_Compressor_MustDecodeWithoutResets

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Shutdown_DebugBreakOnLeaks
Navigation
 OodleX_Shutdown_LogLeaks
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleX_Shutdown_DebugBreakOnLeaks
{
    OodleX_Shutdown_DebugBreakOnLeaks_No = 0,
    OodleX_Shutdown_DebugBreakOnLeaks_Yes = 1,
    OodleX_Shutdown_DebugBreakOnLeaks_Force32 = 0x40000000
};
Discussion
bool enum
 
OodleX_Shutdown_LogLeaksOodleX Startup and ShutdownOodleXInitOptions

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Forming Packets for Maximum Compression
Navigation
 Capturing Training data for OodleNetwork
 About Oodle Network Compression
 Welcome to Oodle
 Change Log

The way you create your network packets can have a large effect on their compressability. These are a few tips for people who are interested in experimenting to get more compression.

What works on different games varies, some of these may not work for your game.

  • Try not to include pre-compressed data. Don't use zlib or any other compressor before giving the data to Oodle. If you do need to send pre-compressed data to the client, don't run those packets through OodleNetwork, and don't include them in the training set.

  • Getting rid of floats is always good. Floating point numbers usually have a lot of useless precision in the low bits that the game doesn't actually care about. These show up as a nearly random byte in the low part of the word. It's best to figure out how much precision you actually need and covert all floats to fixed point integers.

  • Do do reductions that are mathematical in nature, which the compressor will never be able to figure out. For example, send matrices as Euler angles, send only 2 components of normalized vectors, etc.

  • Don't worry too much about bit packing or entropy reduction that is purely statistical. This is what the Oodle Network compressor is good at.

  • Consider turning off bit packing entirely. In some cases it's better to just do byte-aligned packet construction and let the compressor handle all the bit packing. Even though the uncompressed packet will be much bigger, it can result in a smaller compressed packet. It also may save a lot of CPU time by avoiding the bit packing. (this varies a lot from game to game)

  • If you do bit-pack, try to byte-align in strategic places. At the very least at the start of each game entity you should flush to byte alignment. Possibly in other places, such as between the property index and the property value.

  • Try to use absolute indices instead of relative indices when refering to properties or other entities. This makes the identifier stable and consistent, so that the compressor can identify repeated uses of the same identifier.

  • If possible, try to include a hint about the property or entity in its first bytes. Generally don't do this if requires adding a byte you weren't already sending. For example if you send an entity index, use the top bits to indicate the type of entity. That way the compressor can see the first bytes and use them to expect what's coming next.

  • Very large and rare packets should usually be compressed with Oodle Data Compression and not piped through the Oodle Network system. For example you might send a huge initial join packet that's 100 kB , then normal game play packets are around 200 bytes. Those are very different things and should be compressed separately.

  • Split packets for MTU after Oodle network, not before.

See also :

http://cbloomrants.blogspot.com/2012/10/10-16-12-thoughts-on-bit-packing.html
 

Capturing Training data for OodleNetworkAbout Oodle Network CompressionNetwork plugins

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_ReadMallocWholeFile_Async
Navigation
 OodleXIOQ_SetInfoByName_Async
 OodleXIOQ_ReadMallocWholeFile_GetResult
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_ReadMallocWholeFile_Async( OodleXIOQFile file,
                                                  OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                                  OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                                  const OodleXHandle * dependencies OODEFAULT( NULL ),
                                                  OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Start a high level IO request to allocate a buffer for a whole file and read it
Parameters
file  the file to act on
autoDelete  (optional) see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

High level IOQ operations are helpers built on the simpler IOQ low level ops.

OodleXIOQ_ReadMallocWholeFile_Async calls OodleXMalloc_IOAligned to allocate a buffer the size of the whole file (aligned up by OODLEX_IO_MAX_ALIGNMENT), and reads the whole file into that buffer.

Get the buffer pointer with OodleXIOQ_ReadMallocWholeFile_GetResult. You must free it.
 

OodleXIOQ_SetInfoByName_AsyncOodleX low level async ioOodleXIOQ_ReadMallocWholeFile_GetResult

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_PrintfError
Navigation
 OodleXLog_Flush
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXLog_PrintfError( OodleXError err );
Discussion
Log a string describing the OodleXError with OodleXLog_Printf
 
OodleXLog_FlushOodleX Debug aidsOodleXLogCallbackRet

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_GetDefaults_Minimal
Navigation
 OodleX_Init_GetDefaults
 OodleX_Init
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleX_Init_GetDefaults_Minimal( OO_U32 oodle_header_version,
                                         OodleXInitOptions * pOptions );
Discussion
Get minimal defaults for OodleXInitOptions , enabling only necessary Oodle systems
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
pOptions  filled with default OodleXInitOptions
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

Fill options such that a minimal part of the Oodle library is imported.

All memory->memory compressors will work.

IO and Threading will be disabled.

Can be used with OodleX_Init_NoThreads or OodleX_Init
 

OodleX_Init_GetDefaultsOodleX Startup and ShutdownOodleX_Init

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
example_lz : Example demonstrating LZ compression and decompression
Navigation
 How to build and use the Oodle examples
 Examples
 example_lz_chart : Example that makes a chart of OodleLZ options
 Welcome to Oodle
 Change Log
Discussion
Oodle example_lz

Use the various LZ compress/decompress API's

API's demonstrated here :

OodleLZ : low level buffer compress/decompress

OodleLZDecoder : streaming decoder

OodleLZ_Async : high level async helpers


#include "../include/oodle2x.h"
#include "ooex.h"

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "make_example_input.h"

#ifdef BUILDING_EXAMPLE_CALLER
#define main example_lz
#endif

//===========================================================
// file names :

static const char * in_name = "oodle_example_input_file";
static const char * out_name = "oodle_example_output_file";

// just make the input buffer global here so all the tests can use it :
static void * in_buffer = NULL;
static OO_SINTa in_size = 0;

//===========================================================

// protos :

void lz_test_1();
void lz_test_2();
void lz_test_4();
void lz_test_9();
void lz_test_10();
void lz_test_11();
void lz_test_12();
void lz_test_13();

//===========================================================

extern "C" int main(int argc,char *argv[])
{
    OodleXLog_Printf_v1("usage: example_lz [in] [out]\n");

    // Init Oodle systems with default options :
    OodleXInitOptions opts;
    if ( ! OodleX_Init_GetDefaults(OODLE_HEADER_VERSION,&opts) )
    {
        fprintf(stderr,"Oodle header version mismatch.\n");
        return 10;
    }
    // change opts here if you like
    if ( ! OodleX_Init(OODLE_HEADER_VERSION,&opts) )
    {
        fprintf(stderr,"OodleX_Init failed.\n");
        return 10;
    }

    if ( argc >= 2 ) in_name = argv[1];
    else make_example_input(in_name);
    
    if ( argc >= 3 ) out_name = argv[2];

    OodleXLog_Printf_v1("lz test %s to %s\n",in_name,out_name);

    // read the input file to the global buffer :
    OO_S64 in_size_64;
    in_buffer = OodleXIOQ_ReadMallocWholeFile_AsyncAndWait(in_name,&in_size_64);
    
    if ( ! in_buffer)
    {
        OodleXLog_Printf_v0("failed to read %s\n",in_name); 
        return 10;
    }
    
    in_size = OodleX_S64_to_SINTa_check( in_size_64 );
    
    lz_test_1();
    lz_test_2();
    lz_test_4();
    lz_test_9();
    lz_test_10();
    lz_test_11();
    lz_test_12();
    lz_test_13();

    OodleXLog_Printf_v1("\ndone.\n");

    OodleXFree_IOAligned(in_buffer);

    //OodleX_Shutdown();
    OodleX_Shutdown(NULL,OodleX_Shutdown_LogLeaks_Yes,0);
    
    //OodleXLog_Printf_v1("press a key\n");
    //fgetc(stdin);
    
    return 0;
}

//=================================================

lz_test_1 :

example of directly calling the simple buffer->buffer compression API's

OodleLZ_Compress OodleLZ_Decompress


void lz_test_1()
{
    OodleXLog_Printf_v0("lz_test_1\n");
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    void * dec_buf = OodleXMalloc( in_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    //---------------------------------------------------
    
    // compress buffer -> buffer :
    
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_VeryFast;
    
    OO_SINTa comp_len = OodleLZ_Compress(OodleLZ_Compressor_Kraken,in_buffer,in_size,comp_buf,level);
    
    OodleXLog_Printf_v1("Kraken compress %d -> %d\n",(int)in_size,(int)comp_len);

    // decompress :

    OO_SINTa dec_len = OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);

    OOEX_ASSERT_ALWAYS( dec_len == in_size );
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    //---------------------------------------------------
    // do it again with another compressor, and custom options :
    
    // compress buffer -> buffer :
    
    OodleLZ_CompressOptions options = * OodleLZ_CompressOptions_GetDefault(OodleLZ_Compressor_Leviathan,level);
    options.spaceSpeedTradeoffBytes = OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT/2; // favor size over decode speed
    
    comp_len = OodleLZ_Compress(OodleLZ_Compressor_Leviathan,in_buffer,in_size,comp_buf,level,&options);
    
    OodleXLog_Printf_v1("Leviathan compress %d -> %d\n",(int)in_size,(int)comp_len);

    // decompress :

    dec_len = OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);

    OOEX_ASSERT( dec_len == in_size );
    OOEX_ASSERT( memcmp(in_buffer,dec_buf,in_size) == 0 );

    //-------------------------------------
    // free buffers :

    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
}

lz_test_2

example of using the OodleLZ_Async_ async helper functions (eg OodleXLZ_Decompress_Wide_Async) this is the simple way to get the best performance

Use the seekChunkReset option on to get a seekable packed stream.


void lz_test_2()
{
    OodleXLog_Printf_v0("lz_test_2\n");

    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    void * dec_buf = OodleXMalloc( in_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    //---------------------------------------------------
    
    OodleLZ_CompressOptions options = * OodleLZ_CompressOptions_GetDefault(OodleLZ_Compressor_Kraken,OodleLZ_CompressionLevel_Normal);
    // turn on seekChunkReset
    //  this makes chunks independent so they can be decompressed in any order (not just linear)
    options.seekChunkReset = true;
    options.seekChunkLen = OodleLZ_MakeSeekChunkLen(in_size,8);

    // with seekChunkReset on, compression will also go in parallel
    //  (actually compression can *always* run in parallel, but seekChunkReset makes it scale more linearly,
    //    and parallelize on a smaller granularity) 
    // use the OodleLZ_Compressor_Kraken compressor
    
    OodleXHandle h = OodleXLZ_Compress_Async(OodleXAsyncSelect_Full,OodleLZ_Compressor_Kraken,in_buffer,in_size,comp_buf,OodleLZ_CompressionLevel_Normal,&options);

    // ... do other game work while compression runs ...
    
    OO_SINTa comp_len = -1; 
    OodleXLZ_Compress_Wait_GetResult(h,&comp_len);

    OodleXLog_Printf_v1("LZ compress %d -> %d\n",(int)in_size,(int)comp_len);

    //-----------------------------------------------------
    // make seek entries :
    //  seek entries allow parallel decompression
    
    OodleLZ_SeekTable * seekTable = OodleLZ_CreateSeekTable(OodleLZSeekTable_Flags_None,options.seekChunkLen,in_buffer,in_size,comp_buf,comp_len);
    OOEX_ASSERT( seekTable != NULL );

    //-----------------------------------------------------

    OodleXHandle dh = OodleXLZ_Decompress_Wide_Async(OodleXAsyncSelect_Full,seekTable,comp_buf,comp_len,dec_buf,in_size,
                            OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,0,0,OodleLZ_PackedRawOverlap_No,0,0);
    
    // ... do other game work while decompression runs ...
    
    OodleXStatus st = OodleX_Wait(dh,OodleXHandleDeleteIfDone_Yes);
    OOEX_ASSERT_ALWAYS( st == OodleXStatus_Done );
    
    //-----------------------------------------------------
    // check :
        
    OOEX_ASSERT( memcmp(in_buffer,dec_buf,in_size) == 0 );

    //-------------------------------------
    // free buffers :

    OodleLZ_FreeSeekTable(seekTable);
    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
}


test_4 :

example of seeking in packed stream and firing per-chunk decompression tasks

Sort of like what OodleXLZ_Decompress_Wide_Async does internally.


void lz_test_4()
{
    OodleXLog_Printf_v0("lz_test_4\n");
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    void * dec_buf = OodleXMalloc( in_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    // set up compress options for independent seek chunks of the smallest allowed size :
    OodleLZ_CompressOptions lzOptions = * OodleLZ_CompressOptions_GetDefault(OodleLZ_Compressor_Kraken,OodleLZ_CompressionLevel_VeryFast);
    lzOptions.seekChunkReset = true;
    //lzOptions.seekChunkLen = OODLELZ_BLOCK_LEN;
    // make a seek chunk len to target 32 chunks :
    lzOptions.seekChunkLen = OodleLZ_MakeSeekChunkLen(in_size,32);
        
    //---------------------------------------------------
    
    // compress buffer -> buffer :
        
    OO_SINTa comp_len = OodleLZ_Compress(OodleLZ_Compressor_Kraken,in_buffer,in_size,comp_buf,OodleLZ_CompressionLevel_VeryFast,&lzOptions);
    
    OodleXLog_Printf_v1("LZ compress %d -> %d\n",(int)in_size,(int)comp_len);
    
    //---------------------------------------------------
    // decompress by seeking and firing async decodes
    
    OO_SINTa maxNumSeeks = (in_size + lzOptions.seekChunkLen-1)/lzOptions.seekChunkLen;
    
    OodleXHandle * handles = (OodleXHandle *) OodleXMalloc( sizeof(OodleXHandle) * maxNumSeeks );
    OOEX_ASSERT( handles != NULL );

    OO_S32 numHandles = 0;
    
    {   
    OO_SINTa dec_pos = 0;
    OO_U8 * comp_ptr = (OO_U8 *)comp_buf;
    OO_SINTa comp_avail = comp_len;

    while(dec_pos < in_size)    
    {
        OO_SINTa dec_chunk_len = OOEX_MIN(lzOptions.seekChunkLen, (in_size-dec_pos) );

        OodleXHandle h = OodleXLZ_Decompress_Narrow_Async(OodleXAsyncSelect_Full,comp_ptr,comp_avail,(OO_U8 *)dec_buf+dec_pos,dec_chunk_len);
        //  OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None, etc
        handles[numHandles++] = h;
        
        OO_SINTa seek_step = OodleLZ_GetCompressedStepForRawStep(comp_ptr,comp_avail,dec_pos,dec_chunk_len);
        
        comp_ptr += seek_step;
        dec_pos += dec_chunk_len;
        
        // wait on handles[numHandles-128] to prevent handle count exceeding handle table size :
        if ( numHandles >= 128 )
        {
            OodleXStatus st = OodleX_WaitAndDelete(handles[numHandles-128]);
            OOEX_ASSERT_ALWAYS( st == OodleXStatus_Done );
        }
    }
    
    }
    
    // ... do other game work while async decomps run ...
    
    OodleXStatus st = OodleX_WaitAll(handles,numHandles,OodleXHandleDeleteIfDone_Yes);
    OOEX_ASSERT_ALWAYS( st == OodleXStatus_Done );

    OodleXFree(handles);


    // check it's okay :
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    //-------------------------------------
    // free buffers :

    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
}


test_9 :

Demonstrate separate block compression & decompression

OodleLZ blocks can be concatenated to form a single valid LZ data stream That means you can just call OodleLZ_Compress on separate blocks and append the output, then decode in one call.

OodleLZ blocks that were made from separate Compress calls will be independent unless you specified dictionary backup in the encode, which makes them depend on previous data.

The rules are :

1. OodleLZ Decompress can be called on invidual blocks (OODLELZ_BLOCK_LEN) if :

they are seek-chunk-reset points, OR if they were made by separate OodleLZ Compress calls OR if the compressor does not carry state across blocks (OodleLZ_Compressor_MustDecodeWithoutResets)

2. OodleLZ Decompress must get the same dictionary as OodleLZ Compress saw no previous dictionary is needed if it's a seek-chunk-reset point (the start of an OodleLZ_Compress call is always a seek reset point, if no dictionary backup is provided to the encoder)


void lz_test_9()
{
    OodleXLog_Printf_v0("lz_test_9\n");

    //---------------------------------------------------
    // split the buffer into two pieces
    //  such that the split point is a valid seek chunk point :
    OO_SINTa block_size = OodleLZ_MakeSeekChunkLen(in_size,2);

    if ( block_size >= in_size )
    {
        // too small to split at seek chunk
        return;
    }
    
    char * in1 = (char *)in_buffer;
    char * in2 = in1 + block_size;
    OO_SINTa len1 = block_size;
    OO_SINTa len2 = in_size - len1;
    
    OodleXLog_Printf_v1("Chunks : %d + %d\n",(int)len1,(int)len2);

    //---------------------------------------------------
    
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    void * dec_buf = OodleXMalloc( in_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    //---------------------------------------------------

    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Fast; 
    OodleLZ_CompressOptions options = * OodleLZ_CompressOptions_GetDefault(compressor,level);
    
    // options does NOT have seek resets by default

    //-----------------------------
    // compress as one part :
    {

    OO_SINTa comp_len = OodleLZ_Compress(compressor,in_buffer,in_size,comp_buf,level,&options);

    OodleXLog_Printf_v1("Whole buffer compress : %d -> %d\n",(int)in_size,(int)comp_len);   

    // normal one part decompression :
    memset(dec_buf,0xEE,in_size);
    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    OO_U8 * comp_end = (OO_U8 *)comp_buf + comp_len;

    //===================================================================

    // can decode in two calls with the full dictionary, but only for compressors that don't carry state :
    if ( ! OodleLZ_Compressor_MustDecodeWithoutResets(compressor) )
    {
        //-------------------------------------------------------------
        // decode as two parts : (len1,len2) :
        memset(dec_buf,0xEE,in_size);

        OO_SINTa dec_comp_len1 = OodleLZ_GetCompressedStepForRawStep(comp_buf,comp_len,0,len1);
                
        OodleLZ_Decompress(comp_buf,comp_len,dec_buf,len1,OodleLZ_FuzzSafe_Yes);

        // decompress second part with dictionary base :

        OodleLZ_Decompress((char *)comp_buf+dec_comp_len1,comp_len-dec_comp_len1,(char *)dec_buf+len1,len2,
                                    OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,dec_buf,in_size);

        OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

        //-------------------------------------------------------------
        // can also decode block by block :
        memset(dec_buf,0xEE,in_size);
        
        // scan comp_ptr through blocks :
        OO_U8 * comp_ptr = (OO_U8 *)comp_buf;
        for(OO_SINTa block_pos=0;block_pos < in_size;block_pos += OODLELZ_BLOCK_LEN)
        {
            OO_SINTa block_len = OOEX_MIN(OODLELZ_BLOCK_LEN, (in_size - block_pos) );
            
            OO_SINTa block_comp_len = OodleLZ_GetCompressedStepForRawStep(comp_ptr,comp_end-comp_ptr,0,block_len);
            
            // decode current block, with window set to whole buffer :
            OO_SINTa got_pos = OodleLZ_Decompress(comp_ptr,block_comp_len,(char *)dec_buf+block_pos,block_len,
                                    OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,dec_buf,in_size);
            
            OOEX_ASSERT_ALWAYS( got_pos == block_pos+block_len );
            
            comp_ptr += block_comp_len;
        }

        OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );
        //-------------------------------------------------------------
    }
    
    }
    //-----------------------------
    // two part compression with overlap :
    // two compress calls, but using the full window, so decompression must use full window as well
    {
    
    OO_SINTa comp_len1 = OodleLZ_Compress(compressor,in1,len1,comp_buf,level,&options,in_buffer);
    OO_SINTa comp_len2 = OodleLZ_Compress(compressor,in2,len2,(char *)comp_buf + comp_len1,level,&options,in_buffer);
    OO_SINTa comp_len = comp_len1 + comp_len2;

    OodleXLog_Printf_v1("Two part compress with overlap : %d -> %d\n",(int)in_size,(int)comp_len);  

    // must decode whole buffer, but can do it in two calls :

    // you can always just do a whole buffer decode here :
    memset(dec_buf,0xEE,in_size);
    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    // or incremental, but with the whole dictionary :
    memset(dec_buf,0xEE,in_size);

    OO_SINTa dec_comp_len1 = OodleLZ_GetCompressedStepForRawStep(comp_buf,comp_len,0,len1);
        
    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,len1,OodleLZ_FuzzSafe_Yes);
        
    OodleLZ_Decompress((char *)comp_buf+dec_comp_len1,comp_len-dec_comp_len1,(char *)dec_buf+len1,len2,
                                OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,dec_buf,in_size);

    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );
    
    }
    //-----------------------------
    // two part no overlap :
    // second compress doesn't use earlier dictionary here
    // so decompression can be done in two pieces with no overlap
    {
    
    OO_SINTa comp_len1 = OodleLZ_Compress(compressor,in1,len1,comp_buf,level,&options);
    OO_SINTa comp_len2 = OodleLZ_Compress(compressor,in2,len2,(char *)comp_buf + comp_len1,level,&options);
    OO_SINTa comp_len = comp_len1 + comp_len2;

    OodleXLog_Printf_v1("Two part compress no overlap : %d -> %d\n",(int)in_size,(int)comp_len); 

    // can decode in two parts :

    // you can always just do a whole buffer decode here :
    memset(dec_buf,0xEE,in_size);
    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    // or incremental :
    memset(dec_buf,0xEE,in_size);
    
    OO_SINTa dec_comp_len1 = OodleLZ_GetCompressedStepForRawStep(comp_buf,comp_len,0,len1);

    // no dictionary backup needed
    // decode in reverse order to simulate random access :

    OodleLZ_Decompress((char *)comp_buf+dec_comp_len1,comp_len-dec_comp_len1,(char *)dec_buf+len1,len2,OodleLZ_FuzzSafe_Yes);
        
    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,len1,OodleLZ_FuzzSafe_Yes);

    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );
    
    }
    //-----------------------------
    // two part no overlap via seek reset :
    // seek reset system is equivalent to splitting Compress calls like the above
    {   
    options.seekChunkReset = true;
    options.seekChunkLen = (OO_S32)block_size;
    
    OodleXLog_Printf_v1("seekChunkLen : %d\n",options.seekChunkLen);

    OO_SINTa comp_len = OodleLZ_Compress(compressor,in_buffer,in_size,comp_buf,level,&options);

    OodleXLog_Printf_v1("Whole buffer compress seek reset : %d -> %d\n",(int)in_size,(int)comp_len); 

    // can decode in two parts :

    // you can always just do a whole buffer decode here :
    memset(dec_buf,0xEE,in_size);

    OodleXLog_Printf_v1("one part : \n");

    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,in_size,OodleLZ_FuzzSafe_Yes);
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    // or incremental :
    memset(dec_buf,0xEE,in_size);
    
    OodleXLog_Printf_v1("two part : \n");

    OO_SINTa dec_comp_len1 = OodleLZ_GetCompressedStepForRawStep(comp_buf,comp_len,0,len1);
    
    OodleXLog_Printf_v1("dec_comp_len1 = %d\n",(int)dec_comp_len1);

    // no dictionary backup needed
    // decode in reverse order to simulate random access :

    OodleLZ_Decompress((char *)comp_buf+dec_comp_len1,comp_len-dec_comp_len1,(char *)dec_buf+len1,len2,OodleLZ_FuzzSafe_Yes);

    OodleLZ_Decompress(comp_buf,comp_len,dec_buf,len1,OodleLZ_FuzzSafe_Yes);

    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );
    }
    
    //=============================================

    OodleXFree(dec_buf);
    OodleXFree(comp_buf);
}


lz_test_10 :

example of using the incremental/streaming decoder

OodleLZDecoder_Create, etc.

this example shows decoding *from* a limited window

outputs into a single buffer

this example simulates using a limited IO buffer for compressed data

it decodes quanta from the available compressed data



void lz_test_10()
{
    OodleXLog_Printf_v0("lz_test_10\n");
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Fast;
    
    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    OO_U8 * comp_buf = (OO_U8 *) OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    OO_U8 * dec_buf = (OO_U8 *)OodleXMalloc( in_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    //---------------------------------------------------
    
    // compress buffer -> buffer :
    
    OO_SINTa comp_len = OodleLZ_Compress(compressor,in_buffer,in_size,comp_buf,level);
    
    OodleXLog_Printf_v1("LZ compress %d -> %d\n",(int)in_size,(int)comp_len);

    //---------------------------------------------------
    // decompress with incremental streaming decoder :
    
    // we're now going to pretend that "comp_buf" is in a file
    //  and we can't read the whole thing
    
    // 64k IO buffer to stress the code :
    //  obviously you would use much larger
    //  must be at least enough for 1 whole compressed quantum (OODLELZ_BLOCK_MAX_COMPLEN = 256k + 2)
    //  (for Kraken / whole-block compressors you probably want 512k io_buffer minimum)
    const OO_SINTa io_buffer_size = OODLELZ_BLOCK_LEN*2;
    OO_U8 * io_buffer = (OO_U8 *) OodleXMalloc_IOAligned(io_buffer_size);
    OOEX_ASSERT( io_buffer != NULL );
    
    OO_SINTa io_buffer_avail = 0; // starts empty
    
    OO_SINTa comp_file_io_pos = 0; // simulated compressed file next read pos
    
    {
    
    // make the Decoder object using on-stack memory :
    //OO_S32 memSize = OodleLZDecoder_MemorySizeNeeded(compressor,in_size);
    
    OodleLZDecoder * decoder = OodleLZDecoder_Create(compressor,in_size,NULL,0);

    OO_SINTa io_buffer_pos = 0;
    OO_U8 * dec_buf_ptr = (OO_U8 *) dec_buf;
    OO_U8 * dec_buf_end = (OO_U8 *) dec_buf + in_size;;

    while(dec_buf_ptr<dec_buf_end)
    {
        // see if we can do a "read" into the io_buffer :
        if ( comp_file_io_pos < comp_len )
        {
            // don't bother with an IO unless I have some minimum amount of room :
            OO_SINTa min_io_size = 16*1024;
            if ( (io_buffer_size - io_buffer_avail) > min_io_size )
            {
                OO_SINTa io_size = OOEX_MIN( (io_buffer_size - io_buffer_avail), (comp_len - comp_file_io_pos) );
                // stress - limit IO size :
                //io_size = OOEX_MIN(io_size,64*1024);
                
                //OodleXLog_Printf_v1("IO read : %d at %d\n",(int)io_size,(int)comp_file_io_pos);
                
                // IO read :
                memcpy(io_buffer+io_buffer_avail,comp_buf+comp_file_io_pos,io_size);
                comp_file_io_pos += io_size;
                io_buffer_avail += io_size;
            }
        }
    
        // ask the Decoder for a partial decode :
        OodleLZ_DecodeSome_Out out;
        
        OO_BOOL ok = OodleLZDecoder_DecodeSome(decoder,&out,
            dec_buf,(dec_buf_ptr - dec_buf),in_size,(dec_buf_end - dec_buf_ptr),
            io_buffer+io_buffer_pos,io_buffer_avail-io_buffer_pos);

        // real usage should check error return conditions
        OOEX_ASSERT_ALWAYS( ok );

        OO_S32 decoded = out.decodedCount;
        OO_S32 comp_used = out.compBufUsed;
                                
        // advance the decoder :
        dec_buf_ptr += decoded;
        io_buffer_pos += comp_used;
        
        OOEX_ASSERT( out.curQuantumCompLen < io_buffer_size );
        
        //OodleXLog_Printf_v1("decoded : %d using %d\n",decoded,comp_used);
        
        if ( decoded == 0 )
        {
            // couldn't decode anything
            // this should only happen because we're near the end of the io buffer
            // and don't have enough compressed data to do antyhing
            OOEX_ASSERT( io_buffer_pos > 0 );
            // slide down the io buffer so it can refill
            OO_SINTa io_buffer_keep = io_buffer_avail - io_buffer_pos;
            memmove(io_buffer,io_buffer+io_buffer_pos,io_buffer_keep);
            io_buffer_pos = 0;
            io_buffer_avail = io_buffer_keep;
        }
    }

    OOEX_ASSERT( comp_file_io_pos == comp_len );

    OodleLZDecoder_Destroy(decoder);

    }
    
    // check it's okay :
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_buf,in_size) == 0 );

    //-------------------------------------
    // free buffers :

    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
    OodleXFree_IOAligned(io_buffer);
}


lz_test_11 :

example of using the incremental/streaming decoder

OodleLZDecoder_Create, etc.

this example simulates using a limited IO buffer for compressed data (like lz_test_10)

Kraken does not currently have a true "sliding window" decoder; it can't wrap around a circular window. This example shows how to simulate a sliding window with the Kraken decoder by sliding down chunks.

It decodes 256k at a time into a 512k window. It decodes into the second half of the window, with the first window filled by the previous decode. After each decode, it memcopies down the data to be used as dictionary for the next block.

---------

In general this method should not be used if you can just decode directly into the output buffer. That's always the best way if possible.

One case where you might want to use this is if your output buffer is in non-cached graphics memory.

---------

The simpler alternative to this is just to reset every 256k block, so there's no dictionary overlap. eg. just set :

options.seekChunkReset = true; options.seekChunkLen = OODLELZ_BLOCK_LEN;

then you can use a 256k decode output window and don't need to memcpy to slide down the dictionary. The disadvantage of resetting is just lower compression.



void lz_test_11()
{
    OodleXLog_Printf_v0("lz_test_11\n");

    //*
    // fast encoder:
    OodleLZ_Compressor compressor = OodleLZ_Compressor_Kraken;
    OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_VeryFast;
    
/ // slowest encoder: OodleLZ_Compressor compressor = OodleLZ_Compressor_Leviathan; OodleLZ_CompressionLevel level = OodleLZ_CompressionLevel_Optimal5; /*

    //---------------------------------------------------
    
    
// minimum size : // we will decode 256k (one "block") at a time // +256k more for dictionary references to preceding data

OO_S32 decode_window_size = 2*OODLELZ_BLOCK_LEN; // OODLELZ_BLOCK_LEN = 256k OO_S32 dictionary_size = OODLELZ_BLOCK_LEN; /

    // more reasonable size :
    // dictionary limit 2M
    // decode in a 3M window, so we do a memcpy after every 1M streamed
    
    OO_S32 decode_window_size = 3*1024*1024; // OODLELZ_BLOCK_LEN = 256k
    OO_S32 dictionary_size = 2*1024*1024;
    
    
    OodleXLog_Printf_v1("dictionary_size : %d, decode_window_size : %d\n",dictionary_size,decode_window_size);
    
    //---------------------------------------------------
    
    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(compressor,in_size);
    OO_U8 * comp_buf = (OO_U8 *) OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    // dec_window is our scratch circular window
    OO_U8 * dec_window = (OO_U8 *)OodleXMalloc( decode_window_size );
    OOEX_ASSERT( dec_window != NULL );
    
    // dec_out_buf is the final output location (perhaps uncached graphics memory)
    OO_U8 * dec_out_buf = (OO_U8 *)OodleXMalloc( in_size );
    OOEX_ASSERT( dec_out_buf != NULL );
    
    // decoderMem is used for the OodleLZ decoder object
    OO_S32 memSize = OodleLZDecoder_MemorySizeNeeded(compressor);
    OO_U8 * decoderMem = (OO_U8 *) OodleXMalloc(memSize);
    
    //---------------------------------------------------
    // compress buffer -> buffer :
    
    // limit dictionarySize so matches can't go out of the decode window :
    OodleLZ_CompressOptions options = * OodleLZ_CompressOptions_GetDefault(compressor,level);
    options.dictionarySize = dictionary_size;
    
    OO_SINTa comp_len = OodleLZ_Compress(compressor,in_buffer,in_size,comp_buf,level,&options);
    
    OodleXLog_Printf_v1("LZ compress %d -> %d\n",(int)in_size,(int)comp_len);

    //---------------------------------------------------
    // decompress with incremental streaming decoder :
    
    // we're now going to pretend that "comp_buf" is in a file
    //  and we can't read the whole thing
    
    // IO buffer must be at least enough for 1 whole quantum (256k + a little bit)
    //  Kraken uses "large block quantum" (256k) not the old 16k quantum
    const OO_SINTa io_buffer_size = (256+63)*1024;
    OO_U8 io_buffer[io_buffer_size];
    
    OO_SINTa io_buffer_avail = 0; // starts empty
    
    OO_SINTa comp_file_io_pos = 0; // simulated compressed file next read pos
    
    {   
    OodleLZDecoder * decoder = OodleLZDecoder_Create(compressor,in_size,decoderMem,memSize);

    OO_SINTa io_buffer_pos = 0;
    OO_U8 * dec_out_ptr = (OO_U8 *) dec_out_buf;
    OO_U8 * dec_out_end = (OO_U8 *) dec_out_buf + in_size;;

    OO_SINTa dec_window_pos = 0;

    while(dec_out_ptr<dec_out_end)
    {
        // see if we can do a "read" into the io_buffer :
        if ( comp_file_io_pos < comp_len )
        {
            OO_SINTa min_io_size = 16*1024;
            if ( (io_buffer_size - io_buffer_avail) > min_io_size )
            {
                OO_SINTa io_size = OOEX_MIN( (io_buffer_size - io_buffer_avail), (comp_len - comp_file_io_pos) );
                
                //OodleXLog_Printf_v1("IO read : %d at %d\n",(int)io_size,(int)comp_file_io_pos);
                
                // IO read :
                memcpy(io_buffer+io_buffer_avail,comp_buf+comp_file_io_pos,io_size);
                comp_file_io_pos += io_size;
                io_buffer_avail += io_size;
            }
        }
    
        // when dec_window_pos reaches end of window :
        if ( (dec_window_pos + OODLELZ_BLOCK_LEN) > decode_window_size )
        {
            OodleXLog_Printf_v1("slide!\n");
        
            // slide down the dictionary for the next block :
            memmove(dec_window,dec_window + dec_window_pos - dictionary_size,dictionary_size);
            dec_window_pos = dictionary_size;
        }
            
        OodleXLog_Printf_v1("decode : at %d in window, %d in output\n",(int)dec_window_pos,(int)(dec_out_ptr - dec_out_buf));
    
        // ask the Decoder for a partial decode :
        OodleLZ_DecodeSome_Out out;
        
        // no need to truncate dec_avail at the end, the "in_size" passed to LZDecoder_Create does this
        //OO_SINTa dec_avail = OOEX_MIN(dec_out_remain,decode_window_size - dec_window_pos);        
        OO_SINTa dec_avail = decode_window_size - dec_window_pos;
        
        OO_BOOL ok = OodleLZDecoder_DecodeSome(decoder,&out,
            dec_window,dec_window_pos,
            in_size, //decode_window_size, // !!
            dec_avail,
            io_buffer+io_buffer_pos,io_buffer_avail-io_buffer_pos,
            OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No);

        // !! = this is a bit funny; we lie about the window size here
        //  if Kraken had a true sliding window decoder (like eg. LZH or LZB16 does)
        //  then you would pass decode_window_size here and let it do the wrapping
        //  but Kraken does not, so we pretend that we are decoding the whole file
        //  so that OodleLZDecoder_DecodeSome doesn't try to use its sliding window path
        //  (which would fail)
        //  the "dec_avail" value prevents DecodeSome from going out of the window
        //  and we simulate the sliding using the memcpy

        // real usage should check error return conditions
        OOEX_ASSERT_ALWAYS( ok );

        OO_S32 decoded = out.decodedCount;
        OO_S32 comp_used = out.compBufUsed;
        
        io_buffer_pos += comp_used;
        
        OOEX_ASSERT( out.curQuantumCompLen < io_buffer_size );
                
        //OodleXLog_Printf_v1("decoded : %d using %d\n",decoded,comp_used);
        
        if ( decoded == 0 )
        {
            // couldn't decode anything
            // this should only happen because we're near the end of the io buffer
            // and don't have enough compressed data to do antyhing
            OOEX_ASSERT( io_buffer_pos > 0 );
            // slide down the io buffer so it can refill
            OO_SINTa io_buffer_keep = io_buffer_avail - io_buffer_pos;
            memmove(io_buffer,io_buffer+io_buffer_pos,io_buffer_keep);
            io_buffer_pos = 0;
            io_buffer_avail = io_buffer_keep;
        }
        else
        {
            // copy out the decoded data :
            OO_U8 * dec_window_ptr = dec_window + dec_window_pos;
            // dec_out_ptr is the final output memory ; eg. perhaps uncached graphics memory
            // "decoded" is always OODLELZ_BLOCK_LEN unless we hit EOF
            memcpy(dec_out_ptr,dec_window_ptr,decoded);
            
            // advance the decoder :
            dec_out_ptr += decoded;
            dec_window_pos += decoded;
        }
    }

    OOEX_ASSERT( comp_file_io_pos == comp_len );
    OOEX_ASSERT( dec_out_ptr == dec_out_end );

    OodleLZDecoder_Destroy(decoder);
    }
    
    // check it's okay :
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,dec_out_buf,in_size) == 0 );

    //-------------------------------------
    // free buffers :

    OodleXFree(decoderMem);
    OodleXFree(comp_buf);
    OodleXFree(dec_out_buf);
    OodleXFree(dec_window);
}


lz_test_12 :

example of directly calling the simple buffer->buffer compression API's using an "in place" buffer

OodleLZ_Compress OodleLZ_Decompress OodleLZ_GetInPlaceDecodeBufferSize


void lz_test_12()
{
    OodleXLog_Printf_v0("lz_test_12\n");
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken,in_size);
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );
    
    //---------------------------------------------------
    
    // compress buffer -> buffer :
    
    OO_SINTa comp_len;
    
    {
    comp_len = OodleLZ_Compress(OodleLZ_Compressor_Kraken,in_buffer,in_size,comp_buf,OodleLZ_CompressionLevel_Fast);
    }
    
    //---------------------------------------------------

    OO_SINTa inplace_size = OodleLZ_GetInPlaceDecodeBufferSize(OodleLZ_Compressor_Kraken,comp_len,in_size);

    OodleXLog_Printf_v1("Kraken compress %d -> %d ; inplace_size = %d , padding = %d\n",
        (int)in_size,(int)comp_len,
        (int)inplace_size,(int)(inplace_size - in_size));

    void * inplace_buf = OodleXMalloc( inplace_size );
    OOEX_ASSERT( inplace_buf != NULL );

    // in game use, you load the compressed data into the *end* of the inplace buffer 
    // simulate the loading by doing a memcpy :
    
    char * inplace_comp_ptr = (char *)inplace_buf;
    inplace_comp_ptr += inplace_size - comp_len;
    
    memcpy(inplace_comp_ptr,comp_buf,comp_len);

    //---------------------------------------------------
        
    // decompress :
    // note the source (inplace_comp_ptr) and dest (inplace_buf) overlap
    //  - the compressed data at inplace_comp_ptr is destroyed by this call

    OO_SINTa dec_len = OodleLZ_Decompress(inplace_comp_ptr,comp_len,inplace_buf,in_size,OodleLZ_FuzzSafe_Yes);

    OOEX_ASSERT_ALWAYS( dec_len == in_size );
    OOEX_ASSERT_ALWAYS( memcmp(in_buffer,inplace_buf,in_size) == 0 );

    //---------------------------------------------------
    
    // free buffers :

    OodleXFree(comp_buf);
    OodleXFree(inplace_buf);
}


//=================================================

lz_test_13 :

example of dictionary-relative compression

This is a technique in which you train a dictionary offline based on typical data, then for each packet you wish to compress, the dictionary can be used a reference to improve compression ratio.

Oodle can do dictionary relative compression by putting the packet buffer to compress in a contiguous buffer immediately following the dictionary.

Then simply use memcpy to move the active packet to the desired memory location.

NOTE that the work space for {dictionary + packet} must be allocated per thread, or mutex controlled (it cannot be shared by simultaneously decoding threads)

For small packets (under 4 KB or so) such as network packets, consider Oodle Network instead.

For large buffers (over 128 KB or so), dictionary-relative compression doesn't help much and isn't recommended.

dictionary-relative compression is most typically useful on data in the 4 - 128 KB range.


void lz_test_13()
{
    OodleXLog_Printf_v0("lz_test_13\n");
    
    //---------------------------------------------------
    
    // pretend that "in_buffer" consists of a trained dictionary + a packet to compress
    
    void * dictionary = in_buffer;
    OO_SINTa dictionary_size = (in_size * 2) /3;
    // dictionary_size must be a multiple of OODLELZ_BLOCK_LEN :
    dictionary_size &= ~(OODLELZ_BLOCK_LEN-1);
    
    void * packet1 = (char *)dictionary + dictionary_size;
    OO_SINTa packet1_size = in_size / 4;
    
    void * packet2 = (char *)packet1 + packet1_size;
    OO_SINTa packet2_size = in_size - packet1_size - dictionary_size;
    
    OodleXLog_Printf_v1("dictionary_size : %d ; packets : %d + %d\n",dictionary_size,packet1_size,packet2_size);
    
    //---------------------------------------------------
    // allocate compressed buffer & decoded buffer of the correct sizes :

    OO_SINTa max_packet_size = OOEX_MAX(packet1_size,packet2_size);

    // room for dictionary + a packet following :
    void * dictionary_and_packet_buf = OodleXMalloc( dictionary_size + max_packet_size );
    OOEX_ASSERT( dictionary_and_packet_buf != NULL );

    // comp buf just for a packet :
    OO_SINTa comp_buf_size = OodleLZ_GetCompressedBufferSizeNeeded(OodleLZ_Compressor_Kraken, max_packet_size );
    void * comp_buf = OodleXMalloc( comp_buf_size );
    OOEX_ASSERT( comp_buf != NULL );

    // room for dictionary + a packet following :
    void * dec_buf = OodleXMalloc( dictionary_size + max_packet_size );
    OOEX_ASSERT( dec_buf != NULL );
    
    //---------------------------------------------------
    
    void * packets[2] = { packet1, packet2 };
    OO_SINTa packet_sizes[2] = { packet1_size, packet2_size };

    // setup work that's done in advance :

    // put dictionary at head of dictionary_and_packet_buf (for encoder) :
    memcpy(dictionary_and_packet_buf,dictionary,dictionary_size);
    void * after_dictionary_ptr = (char *)dictionary_and_packet_buf + dictionary_size;
    
    // preload dictionary at head of dec_buf (for decoder) :
    memcpy(dec_buf,dictionary,dictionary_size);
    
    for(int packet_i=0;packet_i<2;packet_i++)
    {
        // work that's done per packet :
    
        void * packet_ptr = packets[packet_i];
        OO_SINTa packet_size = packet_sizes[packet_i];
    
        // compress packet to comp_buf
        // preload with dictionary
        
        // copy packet to be immediately following dictionary :
        memcpy(after_dictionary_ptr,packet_ptr,packet_size);
        
        OO_SINTa comp_len = OodleLZ_Compress(OodleLZ_Compressor_Kraken,after_dictionary_ptr,packet_size,comp_buf,OodleLZ_CompressionLevel_Fast,NULL,dictionary_and_packet_buf);
        
        OodleXLog_Printf_v1("Kraken compress %d -> %d\n",(int)packet_size,(int)comp_len);

        // decompress :
        // decode into buffer containing dictionary, immediately following dictionary :

        void * dec_packet_ptr = (char *)dec_buf + dictionary_size;

        OO_SINTa dec_len = OodleLZ_Decompress(comp_buf,comp_len,dec_packet_ptr,packet_size,OodleLZ_FuzzSafe_Yes,OodleLZ_CheckCRC_No,OodleLZ_Verbosity_None,dec_buf);

        OOEX_ASSERT_ALWAYS( dec_len == packet_size + dictionary_size );
        OOEX_ASSERT_ALWAYS( memcmp(packet_ptr,dec_packet_ptr,packet_size) == 0 );
        
        // if you need the decoded packet to be in another memory location, memcpy it there now
    }
    
    //---------------------------------------------------
    // free buffers :

    OodleXFree(dictionary_and_packet_buf);
    OodleXFree(comp_buf);
    OodleXFree(dec_buf);
}

 
How to build and use the Oodle examplesExamplesexample_lz_chart : Example that makes a chart of OodleLZ options

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on WASM
Navigation
 About Oodle on Android
 About Oodle on Platforms
 Welcome to Oodle
 Change Log

Oodle on WASM is currently provided as only the Oodle Core lib (no OodleX).

This includes the synchronous LZ compressors. For WASM, the library tries to have zero runtime dependencies. Among other things, that means that unlike other Oodle platforms, it does not have a default memory allocator or logging implementation.

The Oodle compressors and decompressors do not require dynamic allocations as long as sufficient "scratch"/"decoder" memory is passed in; refer to the documentation for OodleLZ_Compress and OodleLZ_Decompress for details. You need to either do this or install a memory allocator via OodleCore_Plugins_SetAllocators. If you do neither, Oodle compression/decompression requests will fail.

Likewise, if you want to see Oodle diagnostic log messages, install a printf plugin via OodleCore_Plugins_SetPrintf.

Finally, Oodle on WASM does not include the Optimal level encoders. When OodleLZ_CompressionLevel_Optimal1 or higher is requested, the encoder falls back to a variant of OodleLZ_CompressionLevel_Normal instead.
 

About Oodle on AndroidAbout Oodle on PlatformsAbout Oodle Job Threading Plugins

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_DecodeSome_Out
Navigation
 OodleLZ_CompressOptions
 OodleLZDecoder
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleLZ_DecodeSome_Out
{
    OO_S32 decodedCount;
    OO_S32 compBufUsed;
    OO_S32 curQuantumRawLen;
    OO_S32 curQuantumCompLen;
};
Discussion
Output value of OodleLZDecoder_DecodeSome
Members
decodedCount  number of uncompressed bytes decoded
compBufUsed  number of compressed bytes consumed
curQuantumRawLen  tells you the current quantum size. you must have at least this much room available in the output buffer to be able to decode anything.
curQuantumCompLen  if you didn't pass in enough data, nothing will decode (decodedCount will be 0), and this will tell you how much is needed

 
OodleLZ_CompressOptionsOodleAPI_LZ_CompressorsOodleLZDecoder

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetErrorEnum
Navigation
 OodleXIOQ_GetErrorDetails
 OodleXIOQ_LogError
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXError OodleXIOQ_GetErrorEnum( OO_U32 code,
                                    OodleXIOQFile file );
Discussion
Convert an OS error code into a text message
Parameters
code  the error code, eg. from OodleXIOQ_GetStatus
file  the file that the error occurred on (or 0 for unknown)
Return Value
return  an OS-neutral OodleXError
Discussion

Converts an OS-specific error code into a platform agnostic error enum. Useful for recognizing common error cases like OodleXError_FileNotFound. Any unusual or platform-specific codes will return OodleXError_Unknown.
 

OodleXIOQ_GetErrorDetailsOodleX low level async ioOodleXIOQ_LogError

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: My Files aren't loading right and I can't track it down
Navigation
 FAQ: What is SINTa? How do I load files bigger than 2 GB?
 Frequently Asked Questions
 Welcome to Oodle
 Change Log

It can be hard to diagnose IO problems when using async IO, because the actual error is removed from the call that caused the error.

Here are some steps that may help :

1. Make your own OodleXInitOptions struct to pass to OodleX_Init so we can change some of the flags. Use OodleX_Init_GetDefaults to fill it out at first. The members OodleXInitOptions:m_OodleInit_Log and OodleXInitOptions:m_OodleInit_ThreadLog should be on by default, don't turn them off.

2. Set OodleXInitOptions:m_OodleInit_IOQ_Log to true ; this will enable all IO to be logged, and you may see what's causing the error.

3. Set OodleXInitOptions:m_OodleInit_IOQ_Threaded to false ; this will disable IO threading so the error will occur at the call site.

4. Set OodleXInitOptions:m_OodleInit_IOQ_BreakOnError to true; this will cause Oodle IO to do a debug break when it encounters an error. If using this, make sure you are linking with the debug build of Oodle, and using the debug dll on Windows (in the "redistdebug" directory). You should now be able to see the stack of the call causing the error, and your log should contain information about the failure.

5. If your app is crashing because of the error and you can't debug, you can set OodleXInitOptions:m_OodleInit_Log_FlushEachWrite ; hopefully this will get the log to flush out some information about the error before the crash. The log file is written to c:\oodlelogs on Windows unless you have set m_OodleInit_Log_FileName.

(see About Oodle on Platforms for per-platform log info, or set the log name yourself in OodleXInitOptions)
 

FAQ: What is SINTa? How do I load files bigger than 2 GB?Frequently Asked Questions 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNet_Plugins_SetAllocators
Navigation
 OodleNet_Plugins_SetJobSystem
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNet_Plugins_SetAllocators( t_fp_OodleNet_Plugin_MallocAligned * fp_OodleMallocAligned,
                                     t_fp_OodleNet_Plugin_Free * fp_OodleFree );
Discussion
Set the function pointers for allocation needed by Oodle2 Core
Discussion
If these are not set, the default implementation on most platforms uses the C stdlib. On Microsoft platforms the default implementation uses HeapAlloc.

These must not be changed once they are set! Set them once then don't change them.

If you want to ensure that Oodle is not doing any allocations, you can call OodleNet_Plugins_SetAllocators(NULL,NULL); If you do that, then any time Oodle needs to allocate memory internally, it will stop the process. It is STRONGLY not recommended that you ship that way. You can verify that Oodle is not allocating, but then leave some fallback allocator installed when you actually ship just in case.

Also note that on many consoles the standard allocation practices may not leave much heap memory for the C stdlib malloc. In this case Oodle may fail to allocate.


 

Network pluginsNetwork pluginsOodleNet_Plugins_SetJobSystem

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNet_Plugins_SetJobSystemAndCount
Navigation
 OodleNet_Plugins_SetJobSystem
 OodleNet_Plugins_SetPrintf
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleNet_Plugins_SetJobSystemAndCount( t_fp_OodleNet_Plugin_RunJob * fp_RunJob,
                                            t_fp_OodleNet_Plugin_WaitJob * fp_WaitJob,
                                            int target_parallelism );
Discussion
Set the function pointers for async job system needed by Oodle2 Core
Parameters
fp_RunJob  pointer to RunJob function
fp_WaitJob  pointer to WaitJob function
target_parallelism  goal of number of jobs to run simultaneously
Discussion

If these are not set, the default implementation runs jobs synchronously on the calling thread.

These must not be changed once they are set! Set them once then don't change them.

target_parallelism allows you to tell Oodle how many Jobs it should try to keep in flight at once. Depending on the operation it may not be able to split work into this many jobs (so fewer will be used), but it will not exceed this count.

For Oodle Data LZ work, typically target_parallelism is usually best at the number of hardware cores not including hyper threads).

For Oodle Texture BCN encoding work, target_parallelism is usually best as the full number of hyper cores.

In some cases you may wish to reduce target_parallelism by 1 or 2 cores to leave some of the CPU free for other work.

For example on a CPU with 16 cores and 32 hardware threads, for LZ work you might set target_parallelism to 15 when calling OodleCorePlugins. For BC7 encoding you might set target_parallelism to 30 when calling OodleTexPlugins.

NOTE : if you are using Oodle Ext, do NOT call this. OodleX_Init will install a job system for Oodle Core. Note OodleX only installs automatically to Oodle Core, not Net or Tex. See example_jobify.cpp for manual plugin.

Replaces deprecated OodleNet_Plugins_SetJobSystem

See About Oodle Job Threading Plugins
 

OodleNet_Plugins_SetJobSystemNetwork pluginsOodleNet_Plugins_SetPrintf

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXConfigValues
Navigation
 OodleXInitOptions
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleXConfigValues
{
    OO_S32 m_Oodle_DefaultIOBufferSize;
    OO_S32 m_Oodle_DefaultWriteReserveSize;
    OO_S32 m_Oodle_MaxSingleIOSize;
    OO_S32 m_OodleIOQStream_MaxReadSize;
    OO_S32 m_OodleIOQStream_MinReadSize;
    OO_S32 m_OodleIOQStream_OffsetAlignment;
    OO_S32 m_Oodle_very_long_wait_seconds;
    OO_S32 m_deprecated_Desired_Parallel_BranchFactor;
    OO_BOOL m_Oodle_OSFileOpen_Default_Read_Buffered;
    OO_BOOL m_Oodle_OSFileOpen_Default_Write_Buffered;
    OO_BOOL m_Oodle_PathsCaseSensitive;
    OO_U32  m_oodle_header_version;
};
Discussion
OodleXConfigValues
Members
m_Oodle_DefaultIOBufferSize  the buffer size to use when none is given
m_Oodle_DefaultWriteReserveSize  default size to reserve in files opened for write, if none is given
m_Oodle_MaxSingleIOSize  the maximum IO size to submit to the system; larger IO's than this are broken into several pieces; this allows other IO's to interleave, and also prevents heavy loads on kernel resources
m_OodleIOQStream_MaxReadSize  IOQStream doesn't read larger than this (unless a client is blocking on needing more than these bytes immediately). Smaller MaxReadSize reduces IOQStream service latency, but also reduces max throughput
m_OodleIOQStream_MinReadSize  IOQStream tries not to read less than this in a single IO op (unless a client is blocking or we're at EOF or the loop point).
m_OodleIOQStream_OffsetAlignment  IOQStream tries to align all its reads this granularity; some platforms are much faster if the position of IO ops are aligned to large sectors (eg. on the PS3 DVD)
m_Oodle_very_long_wait_seconds  seconds to consider "very long" and warn about possible deadlock
m_deprecated_Desired_Parallel_BranchFactor  number of buffer splits for parallel compress
m_Oodle_OSFileOpen_Default_Read_Buffered  should files opened with OodleXFileOpenFlags_Default for Read be buffered or not?
m_Oodle_OSFileOpen_Default_Write_Buffered  should files opened with OodleXFileOpenFlags_Default for Write be buffered or not?
m_Oodle_PathsCaseSensitive  are paths compared case-sensitive or not? Defaults to the per-platform value OODLEX_PLATFORM_CASE_SENSITIVE.
m_oodle_header_version  = OODLE_HEADER_VERSION
Discussion
Struct of user-settable low level config values. See OodleX_SetConfigValues.

May have different defaults per platform.
 

OodleXInitOptionsOodleX Startup and ShutdownOodleX_Init_ThreadProfilerInit

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_SetVTable
Navigation
 OodleXIOQ_GetOSHandle
 OodleXIOQ_Fence_Async
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleXFileOpsVTable * OodleXIOQ_SetVTable( OodleXIOQFile file,
                                                 const OodleXFileOpsVTable * vtable );
Discussion
Set the VTable used for ops the file.
Parameters
file  the IOQFile to query
Return Value
return  the previous vtable
Discussion

Change the VTable used for ops the file after opening. This is discouraged, generally try to set the right vtable in the OodleXIOQ_OpenForRead_Async call and then don't change it. Warning : vtables are not themselves internally mutex protected ! WARNING : changing the file's VTable while there are ops on that file in the Queue has undefined results !!


 

OodleXIOQ_GetOSHandleOodleX low level async ioOodleXIOQ_Fence_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: Which OodleLZ should I use?
Navigation
 FAQ: How does OodleLZ compare with other compressors ?
 Frequently Asked Questions
 FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?
 Welcome to Oodle
 Change Log

The primary Oodle compressors you might want to use are : Kraken, Mermaid, Selkie, and Leviathan. They are designed for slightly different usage scenarios. This is a simple guide to choosing one.

(See also About OodleLZ for more detailed information about OodleLZ)

The best way to sample them is just to run example_lz_chart : Example that makes a chart of OodleLZ options on some of your data.

All the Oodle compressors support seeking in packed streams and parallel decompression, if you set up the OodleLZ_CompressOptions to request that. (set seekChunkReset)

Kraken is a great general purpose place to start. It has high compression and great decode speed.

Mermaid & Selkie are some of the fastest decompressors in the world. They provide less compression than Kraken but super fast decodes. Selkie is faster than LZ4.

Leviathan offers the highest compression. It's comparable to LZMA (7zip), but a lot faster to decode!

Again :

  • Kraken should be your default choice. For good compression and good decode speed, use Kraken.

  • For more compression, use Leviathan (and OodleLZ_CompressionLevel_Optimal1 or higher)

  • For maximum decompression speed, use Selkie. For very fast decoding with more compression, use Mermaid. (Mermaid is between Selkie and Kraken)

  • Kraken, Mermaid, Selkie and Leviathan should normally be your first choice, but they do better on larger files and may not do well on very small files.

  • For fast encoding, use Mermaid or Selkie with OodleLZ_CompressionLevel_Fast or OodleLZ_CompressionLevel_VeryFast or OodleLZ_CompressionLevel_SuperFast. Kraken, Mermaid & Selkie now also have HyperFast levels with even faster encoding and less compression.

  • Unless you know that you specifically need a small sliding window or seeking, you should compress buffers in the largest pieces possible. In general, cutting data into pieces (via seekChunkReset or your own cutting) reduces compression ratio.

See also FAQ: What are the speeds and ratios of the OodleLZ compressors and levels?.
 

FAQ: How does OodleLZ compare with other compressors ?Frequently Asked QuestionsFAQ: What are the speeds and ratios of the OodleLZ compressors and levels?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_Shared
Navigation
 OodleNetwork1TCP_State
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleNetwork1_Shared;
Discussion
Opaque data type for OodleNetwork1_Shared
Discussion
This data is made from the shared static dictionary. After it is made it is const, and can be used by all compression channels.

This data is filled by OodleNetwork1_Shared_SetWindow.

You can allocate and free it yourself. It must be of size OodleNetwork1_Shared_Size.

Your server must have one of these, and each client must have the exact same one.
 

OODLENETWORK1_DECOMP_BUF_OVERREAD_LENOodleAPI_OodleNetwork1OodleNetwork1TCP_State

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1_CompressedBufferSizeNeeded
Navigation
 OodleNetwork1TCP_State_Size
 OodleNetwork1_Shared_SetWindow
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1_CompressedBufferSizeNeeded( OO_SINTa rawLen );
Discussion
Returns the size of memory required for the compressed buffer passed to OodleNetwork1TCP_Encode
 
OodleNetwork1TCP_State_SizeOodleAPI_OodleNetwork1OodleNetwork1_Shared_SetWindow

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_GetOSHandle
Navigation
 OodleXIOQ_LogLastError
 OodleXIOQ_SetVTable
 Welcome to Oodle
 Change Log
// Function prototype:
void * OodleXIOQ_GetOSHandle( OodleXIOQFile file );
Discussion
Get the OS file handle for this OodleXIOQFile
Parameters
file  the IOQFile to query
Return Value
return  the OS file handle
Discussion

If the file is not yet open (eg. OodleXIOQ_OpenForRead_Async was started but is still pending), this returns NULL.
 

OodleXIOQ_LogLastErrorOodleX low level async ioOodleXIOQ_SetVTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILE_SIZE_INVALID
Navigation
 OODLEX_FILEINFO_MODTIME_INVALID
 OODLEX_FILE_OPEN_NO_RESERVE_SIZE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_FILE_SIZE_INVALID ((OO_S64)-1)
Discussion
Unknown or failure retreiving file size
 
OODLEX_FILEINFO_MODTIME_INVALIDOodleX low level async ioOODLEX_FILE_OPEN_NO_RESERVE_SIZE

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1UDP_State_Compact
Navigation
 OodleNetwork1UDP_State_Compact_ForVersion
 OodleNetwork1UDP_State_Uncompact_ForVersion
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleNetwork1UDP_State_Compact( OodleNetwork1UDP_StateCompacted * to,
                                         const OodleNetwork1UDP_State * from );
Discussion
Fills a OodleNetwork1UDP_StateCompacted from a OodleNetwork1UDP_State
Parameters
to  filled out
from  read
Return Value
return  number of bytes filled in to
Discussion

Use this when the OodleNetwork1UDP_State is created to make a Compacted state to save to a file.

to should be allocated to at least OodleNetwork1UDP_StateCompacted_MaxSize

Note - use the return value to save only the prefix of the Compacted state.


 

OodleNetwork1UDP_State_Compact_ForVersionOodleAPI_OodleNetwork1OodleNetwork1UDP_State_Uncompact_ForVersion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLog_StateFlags
Navigation
 OodleXLog_VerboseLevel
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXLog_StateFlags
{
    OODLEXLOG_TO_FILE = ((OO_U32)1<<0),
    OODLEXLOG_ECHO = ((OO_U32)1<<1),
    OODLEXLOG_TO_DEBUGGER = ((OO_U32)1<<2),
    OODLEXLOG_FILE_LINE = ((OO_U32)1<<3),
    OODLEXLOG_CALLBACK = ((OO_U32)1<<4),
    OODLEXLOG_PREFIX_THREAD_TIME = ((OO_U32)1<<5),
    OODLEXLOG_AUTOFLUSH_THREADLOG = ((OO_U32)1<<6),
    OODLEXLOG_FLUSH_EVERY_WRITE = ((OO_U32)1<<7),
    OODLEXLOG_STATE_VERBOSITY_NONE = ((OO_U32)0<<16),
    OODLEXLOG_STATE_VERBOSITY0 = ((OO_U32)1<<16),
    OODLEXLOG_STATE_VERBOSITY1 = ((OO_U32)2<<16),
    OODLEXLOG_STATE_VERBOSITY2 = ((OO_U32)3<<16)
};
Discussion
Flags for use with OodleXLog_SetState
Enumerants
OODLEXLOG_TO_FILE  log to the log file
OODLEXLOG_ECHO  echo to a stdio file (stdout/stderr typically)
OODLEXLOG_TO_DEBUGGER  log to the debugger
OODLEXLOG_FILE_LINE  put file & line on all logs
OODLEXLOG_CALLBACK  log to the user-provided callback
OODLEXLOG_PREFIX_THREAD_TIME  prefix the thread id & time
OODLEXLOG_AUTOFLUSH_THREADLOG  flush the threadlog to the primary log automatically
OODLEXLOG_FLUSH_EVERY_WRITE  flush log file after every write, useful for debugging crashes
OODLEXLOG_STATE_VERBOSITY_NONE  verbosity in state
OODLEXLOG_STATE_VERBOSITY0 
OODLEXLOG_STATE_VERBOSITY1 
OODLEXLOG_STATE_VERBOSITY2 

 
OodleXLog_PrintfOodleX Debug aidsOodleXLog_VerboseLevel

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Verbosity
Navigation
 OodleLZ_Compressor
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleLZ_Verbosity
{
    OodleLZ_Verbosity_None = 0,
    OodleLZ_Verbosity_Minimal = 1,
    OodleLZ_Verbosity_Some = 2,
    OodleLZ_Verbosity_Lots = 3,
    OodleLZ_Verbosity_Force32 = 0x40000000
};
Discussion
Verbosity of LZ functions LZ functions print information to the function set by OodleCore_Plugins_SetPrintf or OodleXLog_Printf if using OodleX.
 
OODLELZ_SCRATCH_MEM_NO_BOUNDOodleAPI_LZ_CompressorsOodleLZ_Compressor

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_WaitJob
Navigation
 t_fp_OodleNet_Plugin_RunJob
 t_fp_OodleNet_Plugin_Printf
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleNet_Plugin_WaitJob )( OO_U64 job_handle,
                void * user_ptr );
Discussion
Function pointer type for OodleNet_Plugins_SetJobSystem
Parameters
job_handle  a job handle returned from RunJob. Never 0.
user_ptr  is passed through from the OodleLZ_CompressOptions.
Discussion

Waits until the job specified by job_handle is done and cleans up any associated resources. Oodle will call WaitJob exactly once for every RunJob call that didn't return 0.

If job_handle was already completed, this should clean it up without waiting.

A handle value should not be reused by another RunJob until WaitJob has been done with that value.

WaitJob will not be called from running jobs. It will be only be called from the original thread that invoked Oodle. If you are running Oodle from a worker thread, ensure that that thread is allowed to wait on other job threads.

See About Oodle Job Threading Plugins
 

t_fp_OodleNet_Plugin_RunJobNetwork pluginst_fp_OodleNet_Plugin_Printf

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_CombinePaths
Navigation
 OodleXUtil_ConvertUTF16ToUTF8
 OodleX_GetOSCwd
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_CombinePaths( const char * base,
                          const char * add,
                          char * into,
                          OO_S32 intoSize );
Discussion
Combine a base and added path to make an absolute path, correctly
Parameters
base  first part of path
add  second part of path
into  result is written here
intoSize  bytes available in into
Discussion

Combines two paths to make an absolute path. Some of the rules followed :

If add is absolute, into = add

If add begins with ":" or "/" , then into is the drive from base and the full path from add

If add does not begin with a path delim and base does not end with a path delim, one is inserted

(in particular : to do a normal path concatenation, add should NOT start with a path delim)

Relative path actions in add like ".." and "." are respected, but not in base.

The funny DOS-style per-drive current directory reference (eg. "c:blah") is respected at the beginning of add.

The string put in into always has only the preferred path delim for this OS (OODLEX_PATH_DELIM).


 

OodleXUtil_ConvertUTF16ToUTF8OodleX UtilsOodleX_GetOSCwd

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX async handle operations
Navigation
 OodleX Memory Allocators
 Oodle2 Ext API Documentation
 OodleX threading util
 Welcome to Oodle
 Change Log

OodleXHandle is the ubiquitous weak reference to an Oodle Async operation.

  •  About OodleXHandle
  •  Defines
    •  OODLEX_ASYNC_HANDLE_INVALID
    •  OODLEX_ASYNC_HANDLE_PENDING
    •  OODLEX_ASYNC_HANDLE_DONE
    •  OODLEX_ASYNC_HANDLE_ERROR
  •  Enumerants
    •  OodleXPriority
    •  OodleXAsyncSelect
    •  OodleXStatus
    •  OodleXHandleAutoDelete
    •  OodleXHandleKickDelayed
    •  OodleXHandleDeleteIfDone
  •  Functions
    •  OodleX_GetStatus
    •  OodleX_Wait
    •  OodleX_WaitAll
    •  OodleX_WaitDoneAllPending
    •  OodleX_SetHandleAutoDelete
    •  OodleX_GetAvailableAsyncSelect
    •  OodleXHandleEvent_Alloc
    •  OodleXHandleEvent_SetDone
    •  OodleXHandleEvent_SetError
    •  OodleXHandleCountdown_Alloc
    •  OodleXHandleCountdown_Decrement
  •  Typedefs
    •  OodleXHandle

 
OodleXMalloc_GetVTable_OSOodle2 Ext API DocumentationAbout OodleXHandle

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About OodleLZ
Navigation
 Core LZ compression
 About OodleLZ ThreadPhased Decode
 Welcome to Oodle
 Change Log

About OodleLZ

OodleLZ consists of two major API groups : OodleAPI_LZ_Compressors for simple synchronous memory to memory compression (like OodleLZ_Compress) (in the Oodle2 Core lib), and OodleXAPI_LZ_Async (in the Oodle2 Ext lib) for helpers that coordinate IO and multi-threaded invocation of the simple compressors (like OodleXLZ_ReadAndDecompress_Wide_Async).

The Oodle LZ compressors are lossless generic data compressors. They offer world-beating decode speed, with good encode speed and compression ratio. They are particularly well suited to binary "structured data" as is typically found in games.

The most basic Oodle LZ APIs (OodleLZ_Compress and OodleLZ_Decompress) just compress and decompress from memory to memory, one whole buffer at a time.

There are several compressors offered in Oodle. They are listed in the OodleLZ_Compressor enum. Each has slightly different tradeoffs in compression ratio, decode speed, and encode speed. Some of them have other special purpose uses, such as sliding windows or incremental encoding. (feel free to contact oodle@radgametools.com if you have an unusual use case and need guidance)

For each choice of compressor, you can also vary the OodleLZ_CompressionLevel. The compression level just varies the encode speed; slower encodes give higher compression ratios. Typically decode speed is unaffected; it's simply trying harder to pack the data as small as possible in the encoder.

You can also dial decode speed vs compression ratio using the OodleLZ_CompressOptions:spaceSpeedTradeoffBytes parameter.

Kraken (OodleLZ_Compressor_Kraken) is the best compressor to try first. It offers superb decode speed with high compression. It's excellent for data loading in game, distribution, and most general purpose uses.

Mermaid & Selkie are some of the fastest decompressors in the world. They provide less compression than Kraken but super fast decodes. Selkie is faster than LZ4 and offers more compression.

Leviathan offers very high compression with still excellent decode speed, 10X faster than LZMA/7zip. Leviathan should be your choice when you need maximum compression and don't mind being 10-50% slower than Kraken.

Hydra is a meta-compressor that selects Leviathan,Kraken, Mermaid or Selkie for you. See About OodleLZ Hydra

The older Oodle compressors are now deprecated and should not be used (see FAQ: What are the Oodle deprecated compressors ?). Stick to the new sea monsters.

Go ahead and try them with the Oodle Examples

All OodleLZ compressors can optionally break their data into "seek chunks" if OodleLZ_CompressOptions:seekChunkReset is set. The granularity is set in OodleLZ_CompressOptions:seekChunkLen , but must be at least 256k (OODLELZ_BLOCK_LEN). You must enable this at compress time. Doing seekChunkReset hurts compression ratio slightly, but makes it possible to seek in the packed data and decompress just a portion without decompressing the whole thing. It also allows parallel decompression. You may also want an OodleLZ_SeekTable , which records the locations of the seek points; it should be created from the compressed buffer at encode time via OodleLZ_CreateSeekTable. This is needed if you want Oodle to parallelize decompression for you via OodleXLZ_ReadAndDecompress_Wide_Async.

(see FAQ: Which OodleLZ should I use? for a guide to choosing a compressor)

(see FAQ: What are the speeds and ratios of the OodleLZ compressors and levels? for a table of performance for the different choices, or run example_lz_chart : Example that makes a chart of OodleLZ options)

The normal OodleLZ_Compress and OodleLZ_Decompress calls require a full buffer to encode or decode (eg. not streaming or incremental), though you can do individual seek chunks or OODLELZ_BLOCK_LEN at a time. There is also a streaming decoder, which can decode incrementally (eg. OodleLZDecoder_Create).

The OodleLZ compressed data does not include any header. You must store the raw size and compressed size yourself. Because OodleLZ data is headerless, it can be concated at seek chunk boundaries. You can compress any multiple of OODLELZ_BLOCK_LEN bytes separately and simply concatenate the compressed buffers together. This is how you can compress a file that's too big to fit in memory in one piece. You can also use this to do your own multi-threaded compression if you don't want to use Oodle's OodleXLZ_Compress_Async. This is demonstrated in example_lz : Example demonstrating LZ compression and decompression lz_test_9.

The OodleLZ compressors can create a CRC of each compressed chunk so that you can verify its integrity before decompressing (you must set OodleLZ_CompressOptions:sendQuantumCRCs). This CRC is not checked in decode by default, but can be enabled via the OodleLZ_CheckCRC option.

Kraken (and other new compressors) can now do parallel decodes without seek points. See About OodleLZ ThreadPhased Decode for more.


How to choose a compressor and options

The best way to choose a compressor is just to try them. An easy way is to run example_lz_chart : Example that makes a chart of OodleLZ options on your data file to get a sample of how the various Oodle LZ options perform.

Different compression algorithms have different tradeoffs in terms of speed, memory use, and compression ratio.

The OodleLZ compressors are intended to be at the sweet spot for in-game decompression on current game platforms. Their speed/ratio tradeoff is designed to make loading compressed data and then decompressing it faster than just loading decompressed data (in most cases; the exact balance depends on the data and how compressible it is).

Some notes on how to tweak the compressor to meet your goals :

First of all, OodleLZ_CompressionLevel controls how much work the compressor does to optimize the stream. This affects compression time (NOTE : encoding time, not decoding time), but makes better streams. The better streams are both smaller and often decompress faster. So if your goal is to have the smallest stream or the fastest to decompress, in all cases you want to use the highest level of OodleLZ_CompressionLevel that you can tolerate (usually Optimal2 is as high as you need to go).

Note the OodleLZ_CompressionLevel you pass to OodleLZ_CompressOptions_GetDefault does not have to match the one you use for running the compression.

In general, always start with OodleLZ_Compressor_Kraken as your first choice.

1. If you want maximum compression :

Use OodleLZ_Compressor_Leviathan . Don't compress small buffers independently; instead append them together and compress them as one chunk. Do not set OodleLZ_CompressOptions:seekChunkReset (make it false).

Dial OodleLZ_CompressOptions:spaceSpeedTradeoffBytes down. Perhaps try 32. This trades off decode speed for smaller compressed files.

2. If you want maximum decompress speed :

Use OodleLZ_Compressor_Selkie.

For threaded decodes, set OodleLZ_CompressOptions:seekChunkReset to true. This ensures that decompression can proceed in parallel.

Make and transmit a OodleLZ_SeekTable table so that the parallel decoder can find its start points without scanning the data. You can do this manually via OodleLZ_CreateSeekTable , or you can use an OOZ or OOP file to do this for you.

There are options in OodleLZ_CompressOptions which can adjust decode speed, such as minMatchLen, dictionarySize, and spaceSpeedTradeOffBytes. Contact Oodle support for advice on these.

3. About manually setting up OodleLZ_CompressOptions :

Normally you should just use OodleLZ_CompressOptions_GetDefault , but if you want the last bit of speed or compression ratio, you can sometimes find some more win by playing with individuals options. (this is particularly true with unusual data that does not fit well with the default heuristics).

  • seekChunkReset : makes chunks independent; hurts compression but allows parallel decompression, as well as seeking.

  • seekChunkLen : sets the length of seek chunks. Generally you want this to be as large as possible, while providing enough chunks to utilitize all worker threads. A good way to set it is to call OodleLZ_MakeSeekChunkLen with a seek point count of 8 or so.

  • spaceSpeedTradeoffBytes : controls the compressors decision about whether it should favor speed of decompression or small size. It has units of bytes per time; roughly it's the number of bytes that must be saved in compression to make a choice that hurts decode time by some fixed unit of time. The default is 256. For maximum speed of decompress you can set spaceSpeedTradeoffBytes to a larger number (512). For maximum compression set spaceSpeedTradeoffBytes to a small number (32). In normal use the values provided by OodleLZ_CompressOptions_GetDefault are appropriate.

  • minMatchLen : can be used to raise the minimum match length of a compressor (you can't make it lower than the compressor's default). Increasing to 6 or 8 (for example) can sometimes be good for compression or decode speed on some types of data, such as image prediction residuals.

  • dictionarySize : can be used to limit the maximum offset, which may be useful for decode speed on devices with small caches or slow RAM, and can be used to prepare data for sliding window decoding.


If you specify OodleLZ_FuzzSafe_Yes then the output buffer will never be exceeded, even on corrupt data. All the new compressors support Fuzz safe decoding, and it doesn't cost any decode speed. I recommend always using OodleLZ_FuzzSafe_Yes


See also example_lz : Example demonstrating LZ compression and decompression , example_lz_simple : Example demonstrating very simple LZ memory->memory compression using only Oodle Core, example_lz_overlap : Example demonstrating parallel overlap with OodleLZ, example_lz_chart : Example that makes a chart of OodleLZ options
 

Core LZ compressionCore LZ compressionAbout OodleLZ ThreadPhased Decode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_GetDefaultFileOps
Navigation
 OodleX_GetOSFileOps
 OodleX_SetDefaultFileOps
 Welcome to Oodle
 Change Log
// Function prototype:
const OodleXFileOpsVTable * OodleX_GetDefaultFileOps( );
Discussion
Return the current OodleXFileOpsVTable
Discussion
Contains the default file ops function vtable that is used whenever no other vtable is provided.

This begins life equal to the vtable of OodleX_GetOSFileOps , but can be changed.

To mutate use OodleX_SetDefaultFileOps
 

OodleX_GetOSFileOpsOodleX low level async ioOodleX_SetDefaultFileOps

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_PATH_DELIM
Navigation
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_PATH_DELIM /\\
Discussion
OODLEX_PATH_DELIM is either forward slash or back slash, whichever is preferred for the current OS.
Discussion
Oodle IO functions are path-delim agnostic. You only need this if you are using non-Oodle OS IO functions.


 

OodleX UtilsOodleX UtilsOodleX_GetExtensionKey

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Linux
Navigation
 About Oodle on Xbox One
 About Oodle on Platforms
 About Oodle on IOS
 Welcome to Oodle
 Change Log

Oodle for Linux is provided as a lib. There are separate libs for 32 and 64 bit Linux :

lib/liboo2corelinux.a
lib/liboo2extlinux.a
lib/liboo2corelinux64.a
lib/liboo2extlinux64.a

lib/liboo2corelinux.so.##
lib/liboo2extlinux.so.##
lib/liboo2corelinux64.so.##
lib/liboo2extlinux64.so.##

where ## is the Oodle API major version number.

Choose either 32 or 64 bit libs. You may use the static lib (.a) or shared library (.so). You may use just oo2core for Oodle Core, or also use oo2ext for OodleX.

You exe should link against the Oodle shared library with its major version number (eg. liboo2corelinux.so.8) to ensure a compatible library is used. You can make a symbolic link from liboo2corelinux.so to liboo2corelinux.so.8 so that -loo2corelinux can be used.

The SO of OodleX also contains Core. If you use the OodleX SO do not use the Core SO !!

The 64-bit build of Oodle is faster, so should be used when possible.

The debug build of the Oodle lib is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems.

oo2core requires libm and libc and libstdc++. oo2core requires glibc version 2.3.4

oo2ext has more system requirements.

OodleX on Linux requires these standard libs to be linked in :

-lm -lpthread -lrt -lstdc++


Oodle on Linux assumes SCHED_OTHER and doesn't try to do anything with thread priorities.


If you don't set a log file name yourself, OodleX for Linux tries to write a log file to "/var/log/oodle/" by default. If you don't want this, you may disable logging or change the log location in OodleXInitOptions. On most systems, Oodle will fail to create the "oodle" subdir in "var/log/" due to lack of permissions; you must use an administrator account to create that dir for Oodle (and make that dir writeable). If Oodle fails to write the log to the desired location, it will instead write it to "." (current directory).
 

About Oodle on Xbox OneAbout Oodle on PlatformsAbout Oodle on IOS

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleAutoDelete
Navigation
 OodleXStatus
 OodleXHandleKickDelayed
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXHandleAutoDelete
{
    OodleXHandleAutoDelete_No = 0,
    OodleXHandleAutoDelete_Yes = 1,
    OodleXHandleAutoDelete_Force32 = 0x40000000
};
Discussion
When you spawn an async task and get an OodleXHandle back to track the task, with a normal OodleXHandleAutoDelete_No handle you have to ensure that the handle is deleted at some point (typically by calling OodleX_Wait with OodleXHandleDeleteIfDone_Yes).
Enumerants
OodleXHandleAutoDelete_No  (default) handle lifetime will be managed by the client
OodleXHandleAutoDelete_Yes  handle will deleted itself when done
OodleXHandleAutoDelete_Force32 
Discussion
Alternative you can make the handle self-deleting by creating it with the OodleXHandleAutoDelete_Yes option. In that case you can still inspect the handle status with OodleX_GetStatus and OodleX_Wait, but when the handle completes and deletes itself, you will get OodleXStatus_Invalid. You cannot detect Done vs. Error cases with an OodleXHandleAutoDelete_Yes handle.
 
OodleXStatusOodleX async handle operationsOodleXHandleKickDelayed

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLogCallbackRetRet
Navigation
 OodleXLog_VerboseLevel
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXLogCallbackRetRet
{
    OodleXLogCallbackRetRet_Continue = 1,
    OodleXLogCallbackRetRet_Terminate = 0,
    OodleXLogCallbackRetRet_Force32 = 0x40000000
};
Discussion
Return value for OodleXLogCallbackRet
Enumerants
OodleXLogCallbackRetRet_Continue  output to other log States
OodleXLogCallbackRetRet_Terminate  suppress further logging of this message
OodleXLogCallbackRetRet_Force32 

 
OodleXLog_VerboseLevelOodleX Debug aidsOodleXLog_SetState

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZDecoder_MakeValidCircularWindowSize
Navigation
 OodleLZDecoder_DecodeSome
 OodleLZ_MakeSeekChunkLen
 Welcome to Oodle
 Change Log
// Function prototype:
OO_S32 OodleLZDecoder_MakeValidCircularWindowSize( OodleLZ_Compressor compressor,
                                                   OO_S32 minWindowSize OODEFAULT( 0 ) );
Discussion
Get a valid "Window" size for an LZ
Parameters
compressor  which compressor you will be decoding
minWindowSize  (optional) minimum size of the window
Discussion

NOTE: circular windows are deprecated as of 2.9.0

Most common usage is OodleLZDecoder_MakeValidCircularWindowSize(0) to get the minimum window size.

Only compressors which pass OodleLZ_Compressor_CanDecodeInCircularWindow can be decoded in a circular window.

WARNING : this is NOT the size to malloc the window! you need to call OodleLZ_GetDecodeBufferSize() and pass in the window size to get the malloc size.
 

OodleLZDecoder_DecodeSomeOodleAPI_LZ_CompressorsOodleLZ_MakeSeekChunkLen

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleEvent_Alloc
Navigation
 OodleX_GetAvailableAsyncSelect
 OodleXHandleEvent_SetDone
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXHandleEvent_Alloc( OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ) );
Discussion
Allocate an OodleXHandle to a simple data-less event
Parameters
autoDelete  (optional) set the OodleXHandleAutoDelete of the handle
Return Value
return  the handle
Discussion

An "event" simply stores a transition from Pending -> Done/Error and can be used to wait on something you can trigger.
 

OodleX_GetAvailableAsyncSelectOodleX async handle operationsOodleXHandleEvent_SetDone

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Decompress
Navigation
 OodleLZ_Compress
 OodleLZDecoder_Create
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_Decompress( const void * compBuf,
                             OO_SINTa compBufSize,
                             void * rawBuf,
                             OO_SINTa rawLen,
                             OodleLZ_FuzzSafe fuzzSafe OODEFAULT( OodleLZ_FuzzSafe_Yes ),
                             OodleLZ_CheckCRC checkCRC OODEFAULT( OodleLZ_CheckCRC_No ),
                             OodleLZ_Verbosity verbosity OODEFAULT( OodleLZ_Verbosity_None ),
                             void * decBufBase OODEFAULT( NULL ),
                             OO_SINTa decBufSize OODEFAULT( 0 ),
                             OodleDecompressCallback * fpCallback OODEFAULT( NULL ),
                             void * callbackUserData OODEFAULT( NULL ),
                             void * decoderMemory OODEFAULT( NULL ),
                             OO_SINTa decoderMemorySize OODEFAULT( 0 ),
                             OodleLZ_Decode_ThreadPhase threadPhase OODEFAULT( OodleLZ_Decode_Unthreaded ) );
Discussion
Decompress a some data from memory to memory, synchronously.
Parameters
compBuf  pointer to compressed data
compBufSize  number of compressed bytes available (must be greater or equal to the number consumed)
rawBuf  pointer to output uncompressed data into
rawLen  number of uncompressed bytes to output
fuzzSafe  (optional) should the decode fail if it contains non-fuzz safe codecs?
checkCRC  (optional) if data could be corrupted and you want to know about it, pass OodleLZ_CheckCRC_Yes
verbosity  (optional) if not OodleLZ_Verbosity_None, logs some info
decBufBase  (optional) if not NULL, provides preceding data to prime the dictionary; must be contiguous with rawBuf, the data between the pointers dictionaryBase and rawBuf is used as the preconditioning data. The exact same precondition must be passed to encoder and decoder. The decBufBase must be a reset point.
decBufSize  (optional) size of decode buffer starting at decBufBase, if 0, rawLen is assumed
fpCallback  (optional) OodleDecompressCallback to call incrementally as decode proceeds
callbackUserData  (optional) passed as userData to fpCallback
decoderMemory  (optional) pre-allocated memory for the Decoder, of size decoderMemorySize
decoderMemorySize  (optional) size of the buffer at decoderMemory; must be at least OodleLZDecoder_MemorySizeNeeded bytes to be used
threadPhase  (optional) for threaded decode; see About OodleLZ ThreadPhased Decode (default OodleLZ_Decode_Unthreaded)
Return Value
return  the number of decompressed bytes output, OODLELZ_FAILED (0) if none can be decompressed
Discussion

Decodes data encoded with any OodleLZ_Compressor.

Note : rawLen must be the actual number of bytes to output, the same as the number that were encoded with the corresponding OodleLZ_Compress size. You must store this somewhere in your own header and pass it in to this call. compBufSize does NOT need to be the exact number of compressed bytes, is the number of bytes available in the buffer, it must be greater or equal to the actual compressed length.

Note that the new compressors (Kraken,Mermaid,Selkie,BitKnit) are all fuzz safe and you can use OodleLZ_FuzzSafe_Yes with them and no padding of the decode target buffer.

If checkCRC is OodleLZ_CheckCRC_Yes, then corrupt data will be detected and the decode aborted. If checkCRC is OodleLZ_CheckCRC_No, then corruption might result in invalid data, but no detection of any error (garbage in, garbage out).

If corruption is possible, fuzzSafe is No and checkCRC is OodleLZ_CheckCRC_No, OodleLZ_GetDecodeBufferSize must be used to allocate rawBuf large enough to prevent overrun.

OodleLZ_GetDecodeBufferSize should always be used to ensure rawBuf is large enough, even when corruption is not possible (when fuzzSafe is No).

compBuf and rawBuf are allowed to overlap for "in place" decoding, but then rawBuf must be allocated to the size given by OodleLZ_GetInPlaceDecodeBufferSize , and the compressed data must be at the end of that buffer.

An easy way to take the next step to parallel decoding is with OodleXLZ_Decompress_MakeSeekTable_Wide_Async (in the Oodle2 Ext lib)

NOTE : the return value is the total number of decompressed bytes output so far. If rawBuf is > decBufBase, that means the initial inset of (rawBuf - decBufBase) is included! (eg. you won't just get rawLen)

If decBufBase is provided, the backup distance from rawBuf must be a multiple of OODLELZ_BLOCK_LEN

About fuzz safety:

OodleLZ_Decompress is guaranteed not to crash even if the data is corrupted when fuzzSafe is set to OodleLZ_FuzzSafe_Yes. When fuzzSafe is Yes, the target buffer (rawBuf and rawLen) will never be overrun. Note that corrupted data might not be detected (the return value might indicate success).

Fuzz Safe decodes will not crash on corrupt data. They may or may not return failure, and produce garbage output.

Fuzz safe decodes will not read out of bounds. They won't put data on the stack or previously in memory into the output buffer.

Fuzz safe decodes will not output more than the uncompressed size. (eg. the output buffer does not need to be padded like OodleLZ_GetDecodeBufferSize)

If you ask for a fuzz safe decode and the compressor doesn't satisfy OodleLZ_Compressor_CanDecodeFuzzSafe then it will return failure.

The fuzzSafe argument should always be OodleLZ_FuzzSafe_Yes as of Oodle 2.9.0 ; older compressors did not support fuzz safety but they now all do.

Use of OodleLZ_FuzzSafe_No is deprecated.


 

OodleLZ_CompressOodleAPI_LZ_CompressorsOodleLZDecoder_Create

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_LogLastError
Navigation
 OodleXIOQ_ClearError
 OodleXIOQ_GetOSHandle
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_LogLastError( OodleXIOQFile file );
Discussion
Log the last error on a file
Parameters
file  the IOQFile to query
Return Value
return  true if any error was logged
Discussion

Calls OodleXIOQ_GetLastError and OodleXIOQ_LogLastError
 

OodleXIOQ_ClearErrorOodleX low level async ioOodleXIOQ_GetOSHandle

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMallocVTable
Navigation
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleXMallocVTable
{
    void * m_context;
    void * (OODLE_CALLBACK *m_pMalloc)( void * context, OO_SINTa bytes );
    void * (OODLE_CALLBACK *m_pMallocAligned)( void * context, OO_SINTa bytes , OO_S32 alignment );
    void   (OODLE_CALLBACK *m_pFree)(void * context, void * ptr);
    void   (OODLE_CALLBACK *m_pFreeSized)(void * context, void * ptr, OO_SINTa bytes);
    OO_S32 m_bigAlignment;
    void * (OODLE_CALLBACK *m_pMallocBig)(void * context, OO_SINTa bytes);
    void   (OODLE_CALLBACK *m_pFreeBig)  (void * context, void * ptr);
    OO_BOOL (OODLE_CALLBACK *m_pValidatePointer)(void * context, void * ptr, OO_SINTa bytes);
};
Discussion

Function pointer table used to install the OodleX memory allocation functions

Members
m_context  provided context pointer will be passed to the function pointers
m_pMalloc  must return OODLE_MALLOC_MINIMUM_ALIGNMENT aligned memory
m_pMallocAligned  alignment will always be power of two
m_pFree  must be able to free pointers from m_pMalloc or m_pMallocAligned
m_pFreeSized  use size to make free faster
m_bigAlignment  indicates the alignment provided by MallocBig ; must be a multiple of OODLEX_IO_MAX_ALIGNMENT for OodleX
m_pMallocBig  must return memory aligned to m_bigAlignment
m_pFreeBig  free a pointer allocated by m_pMallocBig
m_pValidatePointer  check on an allocation
Discussion

Use OodleXMalloc_InstallVTable to register a vtable as the one you want OodleX to use. More commonly let OodleX_Init set one for you.
 

OodleXMalloc_OS_OptionsOodleX Memory AllocatorsOodleXMalloc_InstallVTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle_UsageWarnings
Navigation
 Welcome to Oodle
 Change Log
// Enumerant:
enum Oodle_UsageWarnings
{
    Oodle_UsageWarnings_Enabled = 0,
    Oodle_UsageWarnings_Disabled = 1,
    Oodle_UsageWarnings_Force32 = 0x40000000
};
Discussion
Whether Oodle usage warnings are enable or disabled.
 
OodleNetworkVersionCore BaseOodleConfigValues

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CreateSeekTable
Navigation
 OodleLZ_FillSeekTable
 OodleLZ_FreeSeekTable
 Welcome to Oodle
 Change Log
// Function prototype:
OodleLZ_SeekTable * OodleLZ_CreateSeekTable( OodleLZSeekTable_Flags flags,
                                             OO_S32 seekChunkLen,
                                             const void * rawBuf,
                                             OO_SINTa rawLen,
                                             const void * compBuf,
                                             OO_SINTa compLen );
Discussion
allocate a table, then scan compressed LZ stream to fill the seek table
Parameters
flags  options
seekChunkLen  the length of a seek chunk (eg from OodleLZ_MakeSeekChunkLen)
rawBuf  (optional) uncompressed buffer; used to compute the rawCRCs member of OodleLZ_SeekTable
rawLen  size of rawBuf
compBuf  compressed buffer
compLen  size of compBuf
Return Value
return  pointer to table if succeeded, null if failed
Discussion

Same as OodleLZ_FillSeekTable , but allocates the memory for you. Use OodleLZ_FreeSeekTable to free.

seekChunkLen must be a multiple of OODLELZ_BLOCK_LEN. seekChunkLen must match what was in CompressOptions when the buffer was made, or any integer multiple thereof.


 

OodleLZ_FillSeekTableOodleAPI_LZ_CompressorsOodleLZ_FreeSeekTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_SetHandleAutoDelete
Navigation
 OodleX_WaitDoneAllPending
 OodleX_GetAvailableAsyncSelect
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleX_SetHandleAutoDelete( OodleXHandle h,
                                         OodleXHandleAutoDelete autoDelete );
Discussion
change handle lifetime management
Parameters
h  OodleXHandle weak reference
autoDelete  if OodleXHandleAutoDelete_Yes, the handle deletes itself when not pending
Discussion

Handles that are OodleXHandleAutoDelete_No must be deleted or they will leak. The normal way to delete them is by calling OodleX_Wait with OodleXHandleDeleteIfDone_Yes .

A handle that deletes itself when done will then report OodleXStatus_Invalid to queries, because it no longer exists.

If you change a handle to OodleXHandleAutoDelete_Yes and it is already done, this function will delete it immediately, and the returned Status will not be OodleXStatus_Pending.
 

OodleX_WaitDoneAllPendingOodleX async handle operationsOodleX_GetAvailableAsyncSelect

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Oodle2 Ext API Documentation
Navigation
 Welcome to Oodle
 Change Log
  •  OodleX LZ compression
    •  OodleXAPI_LZ_Async
      •  Structures
        •  OodleDecompressCallback_WriteFile_Data
      •  Functions
        •  OodleXLZ_Decompress_ThreadPhased_Narrow_Async
        •  OodleXLZ_Decompress_Narrow_Async
        •  OodleXLZ_ReadAndDecompress_Wide_Async
        •  OodleXLZ_Decompress_Wide_Async
        •  OodleXLZ_Decompress_MakeSeekTable_Wide_Async
        •  OodleXLZ_Compress_Async
        •  OodleXLZ_Compress_Wait_GetResult
        •  OodleXLZ_Compress_AsyncAndWait
        •  OodleXLZ_ReadAndDecompress_Stream_Async
        •  OodleXDecompressCallback_WriteFile_Data_Init
        •  OodleDecompressCallback_WriteFile
  •  OodleX Startup and Shutdown
    •  Defines
      •  OODLE_WORKERS_COUNT_ALL_PHYSICAL_CORES
      •  OODLE_WORKERS_COUNT_ALL_HYPER_CORES
    •  Enumerants
      •  OodleX_Init_GetDefaults_DebugSystems
      •  OodleX_Init_GetDefaults_Threads
      •  OodleX_Shutdown_LogLeaks
      •  OodleX_Shutdown_DebugBreakOnLeaks
    •  Structures
      •  OodleXInitOptions
      •  OodleXConfigValues
    •  Functions
      •  OodleX_Init_ThreadProfilerInit
      •  OodleX_Init_GetDefaults
      •  OodleX_Init_GetDefaults_Minimal
      •  OodleX_Init
      •  OodleX_Init_Default
      •  OodleX_LogSystemInfo
      •  OodleX_Shutdown
      •  OodleX_Init_NoThreads
      •  OodleX_Shutdown_NoThreads
      •  OodleX_GetConfigValues
      •  OodleX_SetConfigValues
  •  OodleX Memory Allocators
    •  About OodleXMalloc
    •  Enumerants
      •  OodleXMalloc_OS_Options
    •  Structures
      •  OodleXMallocVTable
    •  Functions
      •  OodleXMalloc_InstallVTable
      •  OodleXMalloc_SetFailedHandler
      •  OodleXMalloc
      •  OodleXMallocAligned
      •  OodleXFree
      •  OodleXFreeSized
      •  OodleXMallocBigAlignment
      •  OodleXMallocBig
      •  OodleXFreeBig
      •  OodleXMalloc_ValidatePointer
      •  OodleXMalloc_IOAligned
      •  OodleXFree_IOAligned
      •  OodleXMalloc_GetVTable_Clib
      •  OodleXMalloc_GetVTable_OS
    •  Typedefs
      •  OodleXMallocFailedHandler
  •  OodleX async handle operations
    •  About OodleXHandle
    •  Defines
      •  OODLEX_ASYNC_HANDLE_INVALID
      •  OODLEX_ASYNC_HANDLE_PENDING
      •  OODLEX_ASYNC_HANDLE_DONE
      •  OODLEX_ASYNC_HANDLE_ERROR
    •  Enumerants
      •  OodleXPriority
      •  OodleXAsyncSelect
      •  OodleXStatus
      •  OodleXHandleAutoDelete
      •  OodleXHandleKickDelayed
      •  OodleXHandleDeleteIfDone
    •  Functions
      •  OodleX_GetStatus
      •  OodleX_Wait
      •  OodleX_WaitAll
      •  OodleX_WaitDoneAllPending
      •  OodleX_SetHandleAutoDelete
      •  OodleX_GetAvailableAsyncSelect
      •  OodleXHandleEvent_Alloc
      •  OodleXHandleEvent_SetDone
      •  OodleXHandleEvent_SetError
      •  OodleXHandleCountdown_Alloc
      •  OodleXHandleCountdown_Decrement
    •  Typedefs
      •  OodleXHandle
  •  OodleX threading util
    •  Functions
      •  OodleX_Semaphore_Post
      •  OodleX_Semaphore_Wait
      •  OodleX_CreateThread
      •  OodleX_WaitAndDestroyThread
      •  OodleX_ReleaseThreadTLS
      •  OodleX_CorePlugin_RunJob
      •  OodleX_CorePlugin_WaitJob
      •  OodleX_GetNumWorkerThreads
    •  Typedefs
      •  OodleX_Semaphore
      •  OodleX_ThreadFunc
  •  OodleX low level async io
    •  About OodleIOQ
    •  Defines
      •  OODLEX_IO_MAX_ALIGNMENT
      •  OODLEX_BUFFER_SIZE_DEFAULT
      •  OODLEX_FILEINFO_FLAG_INVALID
      •  OODLEX_FILEINFO_MODTIME_INVALID
      •  OODLEX_FILE_SIZE_INVALID
      •  OODLEX_FILE_OPEN_NO_RESERVE_SIZE
      •  OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
    •  Enumerants
      •  OodleXCopyFileFlags
      •  OodleFileNotFoundIsAnError
      •  OODLEX_FILEINFO_FLAGS
      •  OodleXError
      •  OodleXFileMode
      •  OodleXFileOpenFlags
    •  Structures
      •  OodleXFileOpsVTable
      •  OodleXFileInfo
    •  Functions
      •  OodleXIOQ_WaitDoneAllPending
      •  OodleXIOQ_KickAnyDelayed
      •  OodleXIOQ_GetStatus
      •  OodleXIOQ_GetErrorDetails
      •  OodleXIOQ_GetErrorEnum
      •  OodleXIOQ_LogError
      •  OodleXIOQ_Wait
      •  OodleXIOQ_GetInfo
      •  OodleXIOQ_Wait_GetInfo
      •  OodleXIOQ_GetLastPendingOpOnFile
      •  OodleXIOQ_GetName
      •  OodleXIOQ_GetLastError
      •  OodleXIOQ_ClearError
      •  OodleXIOQ_LogLastError
      •  OodleXIOQ_GetOSHandle
      •  OodleXIOQ_SetVTable
      •  OodleXIOQ_Fence_Async
      •  OodleXIOQ_OpenForRead_Async
      •  OodleXIOQ_OpenAndRead_Async
      •  OodleXIOQ_OpenForWriteCreate_Async
      •  OodleXIOQ_OpenForWriteTempName_Async
      •  OodleXIOQ_CloseFile_Async
      •  OodleXIOQ_CloseFileRename_Async
      •  OodleXIOQ_Read_Async
      •  OodleXIOQ_Write_Async
      •  OodleXIOQ_SetFileSize_Async
      •  OodleXIOQ_ReserveFileSizeForWrite_Async
      •  OodleXIOQ_ForceWriteable_Async
      •  OodleXIOQ_Delete_Async
      •  OodleXIOQ_Rename_Async
      •  OodleXIOQ_MakeDir_Async
      •  OodleXIOQ_FreeBufferIOAligned_Async
      •  OodleXIOQ_GetInfoByName_Async
      •  OodleXIOQ_GetInfoByName_GetResult
      •  OodleXIOQ_SetInfoByName_Async
      •  OodleXIOQ_ReadMallocWholeFile_Async
      •  OodleXIOQ_ReadMallocWholeFile_GetResult
      •  OodleXIOQ_OpenAndReadMallocWholeFile_Async
      •  OodleXIOQ_OpenAndReadMallocWholeFileAndClose_Async
      •  OodleXIOQ_OpenWriteWholeFileClose_Async
      •  OodleXIOQ_OpenWriteWholeFileCloseTempName_Async
      •  OodleXIOQ_ReadUnalignedAdjustPointer_Async
      •  OodleXIOQ_MakeAllDirs_Async
      •  OodleXIOQ_CopyFile_Async
      •  OodleXIOQ_ReadMallocWholeFile_AsyncAndWait
      •  OodleXIOQ_WriteWholeFile_AsyncAndWait
      •  OodleXIOQ_CopyFile_AsyncAndWait
      •  OodleXIOQ_GetInfoByName_AsyncAndWait
      •  OodleXIOQ_SetInfoByName_AsyncAndWait
      •  OodleXIOQ_MakeAllDirs_AsyncAndWait
      •  OodleXIOQ_Delete_AsyncAndWait
      •  OodleXIOQ_Rename_AsyncAndWait
      •  OodleXIOQ_GetFileSize_AsyncAndWait
      •  OodleXIOQ_NameIsDir_AsyncAndWait
      •  OodleX_GetOSFileOps
      •  OodleX_GetDefaultFileOps
      •  OodleX_SetDefaultFileOps
    •  Typedefs
      •  OodleXIOQFile
      •  OodleXOSFile
      •  OodleXOSFileListing
  •  OodleX Debug aids
    •  Defines
      •  OodleXLog_Printf
    •  Enumerants
      •  OodleXLog_StateFlags
      •  OodleXLog_VerboseLevel
      •  OodleXLogCallbackRetRet
    •  Functions
      •  OodleXLog_SetState
      •  OodleXLog_SetEcho
      •  OodleXLog_GetEcho
      •  OodleXLog_SetCallback
      •  OodleXLog_GetCallback
      •  OodleXLog_GetVerboseLevel
      •  OodleXLog_SetVerboseLevel
      •  OodleXLog_Flush
      •  OodleXLog_PrintfError
    •  Typedefs
      •  OodleXLogCallbackRet
  •  OodleX Utils
    •  Defines
      •  OODLEX_PATH_DELIM
    •  Functions
      •  OodleX_GetExtensionKey
      •  OodleX_MakeExtensionKey
      •  OodleX_IOAlignUpS32
      •  OodleX_IOAlignUpS64
      •  OodleX_IOAlignUpSINTa
      •  OodleX_IOAlignDownS32
      •  OodleX_IOAlignDownS64
      •  OodleX_IOAlignDownSINTa
      •  OodleX_S64_to_SINTa_check
      •  OodleX_GetSeconds
      •  OodleXUtil_ConvertUTF8ToUTF16
      •  OodleXUtil_ConvertUTF16ToUTF8
      •  OodleX_CombinePaths
      •  OodleX_GetOSCwd
      •  OodleX_PrefixOSCwd
  •  About Oodle Ext

 
  OodleX LZ compression

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXFreeBig
Navigation
 OodleXMallocBig
 OodleXMalloc_ValidatePointer
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXFreeBig( void * ptr );
Discussion
free a pointer allocated by OodleXMallocBig
Parameters
ptr  pointer to free (must not be NULL)
Discussion

You cannot call OodleXFree on a pointer allocated by OodleXMallocBig. Uses the current OodleXMallocVTable ; this is an error if ptr was allocated from a different VTable.
 

OodleXMallocBigOodleX Memory AllocatorsOodleXMalloc_ValidatePointer

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_Default
Navigation
 OodleX_Init
 OodleX_LogSystemInfo
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleX_Init_Default( OO_U32 oodle_header_version,
                             OodleX_Init_GetDefaults_DebugSystems debugSystems OODEFAULT( OodleX_Init_GetDefaults_DebugSystems_Yes ),
                             OodleX_Init_GetDefaults_Threads threads OODEFAULT( OodleX_Init_GetDefaults_Threads_Yes ) );
Discussion
Initialize Oodle, without options struct
Parameters
oodle_header_version  pass OODLE_HEADER_VERSION here
debugSystems  should OodleX_Init enable any debug systems (leaktrack, log, etc) ?
threads  should OodleX_Init start any threads?
Return Value
return  false if OODLE_HEADER_VERSION is not compatible with this lib
Discussion

The debugSystems and threads options are just easy ways of getting pOptions filled out for common use cases. For fine control of individual settings, you can always set the values in OodleXInitOptions yourself.

This is just a shortcut to OodleX_Init_GetDefaults then OodleX_Init

NOTE : do not use this if you want minimal linkage.
 

OodleX_InitOodleX Startup and ShutdownOodleX_LogSystemInfo

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLZ_Compress_Wait_GetResult
Navigation
 OodleXLZ_Compress_Async
 OodleXLZ_Compress_AsyncAndWait
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXStatus OodleXLZ_Compress_Wait_GetResult( OodleXHandle h,
                                               OO_SINTa * pCompLen );
Discussion
Wait, get result, and delete the handle
Parameters
h  the handle from OodleXLZ_Compress_Async
pCompLen  filled with the compressed len
Return Value
return  OodleXStatus_Done for success
Discussion


 

OodleXLZ_Compress_AsyncOodleXAPI_LZ_AsyncOodleXLZ_Compress_AsyncAndWait

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetSeekTableMemorySizeNeeded
Navigation
 OodleLZ_GetNumSeekChunks
 OodleLZ_FillSeekTable
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetSeekTableMemorySizeNeeded( OO_S32 numSeekChunks,
                                               OodleLZSeekTable_Flags flags );
Discussion
Tells you the size in bytes to allocate the seekTable before calling OodleLZ_FillSeekTable
Parameters
numSeekChunks  number of seek chunks (eg from OodleLZ_GetNumSeekChunks)
flags  options that will be passed to OodleLZ_CreateSeekTable
Return Value
return  size in bytes of memory needed for seek table
Discussion

If you wish to provide the memory for the seek table yourself, you may call this to get the required size, allocate the memory, and then simply point a OodleLZ_SeekTable at your memory. Then use OodleLZ_FillSeekTable to fill it out.

Do NOT use sizeof(OodleLZ_SeekTable) !
 

OodleLZ_GetNumSeekChunksOodleAPI_LZ_CompressorsOodleLZ_FillSeekTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_Compressor_CanDecodeThreadPhased
Navigation
 OodleLZ_Compressor_CanDecodeInCircularWindow
 OodleLZ_Compressor_CanDecodeInPlace
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleLZ_Compressor_CanDecodeThreadPhased( OodleLZ_Compressor compressor );
Discussion
OodleLZ_Compressor properties helper.
Discussion
Tells you if this compressor can be used with the OodleLZ_Decode_ThreadPhase.

See About OodleLZ ThreadPhased Decode
 

OodleLZ_Compressor_CanDecodeInCircularWindowOodleAPI_LZ_CompressorsOodleLZ_Compressor_CanDecodeInPlace

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_OodleFPVoidVoidStar
Navigation
 t_OodleFPVoidVoid
 Welcome to Oodle
 Change Log
// Function typedef:
void( OODLE_CALLBACK t_OodleFPVoidVoidStar )( void * );
Discussion
void-void-star callback func pointer takes void pointer, returns void
 
t_OodleFPVoidVoidCore BaseCore plugins

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLELZ_BLOCK_LEN
Navigation
 OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULT
 OODLELZ_BLOCK_MAX_COMPLEN
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLELZ_BLOCK_LEN (1<<18)
Discussion
The number of raw bytes per "seek chunk" Seek chunks can be decompressed independently if OodleLZ_CompressOptions:seekChunkReset is set.
 
OODLELZ_SPACESPEEDTRADEOFFBYTES_DEFAULTOodleAPI_LZ_CompressorsOODLELZ_BLOCK_MAX_COMPLEN

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXAPI_LZ_Async
Navigation
 OodleX LZ compression
 Welcome to Oodle
 Change Log

Async dispatchers and high level helpers for OodleLZ

  •  Structures
    •  OodleDecompressCallback_WriteFile_Data
  •  Functions
    •  OodleXLZ_Decompress_ThreadPhased_Narrow_Async
    •  OodleXLZ_Decompress_Narrow_Async
    •  OodleXLZ_ReadAndDecompress_Wide_Async
    •  OodleXLZ_Decompress_Wide_Async
    •  OodleXLZ_Decompress_MakeSeekTable_Wide_Async
    •  OodleXLZ_Compress_Async
    •  OodleXLZ_Compress_Wait_GetResult
    •  OodleXLZ_Compress_AsyncAndWait
    •  OodleXLZ_ReadAndDecompress_Stream_Async
    •  OodleXDecompressCallback_WriteFile_Data_Init
    •  OodleDecompressCallback_WriteFile

 
OodleX LZ compressionOodleX LZ compressionOodleDecompressCallback_WriteFile_Data

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLEX_FILE_OPEN_NO_RESERVE_SIZE
Navigation
 OODLEX_FILE_SIZE_INVALID
 OODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLEX_FILE_OPEN_NO_RESERVE_SIZE (0)
Discussion
Pass for reserveSize to OpenFile calls if you don't want it to reserve any space
 
OODLEX_FILE_SIZE_INVALIDOodleX low level async ioOODLEX_FILE_CLOSE_NO_TRUNCATE_SIZE

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXError
Navigation
 OODLEX_FILEINFO_FLAGS
 OodleXFileMode
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXError
{
    OodleXError_Ok = 0,
    OodleXError_InvalidHandle = 1,
    OodleXError_FileNotFound = 2,
    OodleXError_NoAccess = 3,
    OodleXError_BadParameters = 4,
    OodleXError_Corrupt = 5,
    OodleXError_Alignment = 6,
    OodleXError_Malloc = 7,
    OodleXError_Compressor = 8,
    OodleXError_UnexpectedEOF = 9,
    OodleXError_PreviousAsyncFailed = 10,
    OodleXError_Close = 11,
    OodleXError_Unknown,
    OodleXError_Count,
    OodleXError_Force32 = 0x40000000
};
Discussion
oodle error enum to get a platform independent simple error code
Enumerants
OodleXError_Ok  no error
OodleXError_InvalidHandle  null pointer, not open file, etc
OodleXError_FileNotFound  file not found
OodleXError_NoAccess  attrib or sharing violation
OodleXError_BadParameters  usually unaligned or out of bounds file pointers
OodleXError_Corrupt  scratch or ejected media, damaged bits
OodleXError_Alignment  wrong alignment
OodleXError_Malloc  alloc failed
OodleXError_Compressor  a compressor or decompressor failed
OodleXError_UnexpectedEOF  eof where I needed data
OodleXError_PreviousAsyncFailed  dependent async failed, so I can't run
OodleXError_Close  error in close or object deletion, so I can't get more info
OodleXError_Unknown  error that doesn't match any of the other enums
OodleXError_Count 
OodleXError_Force32 

 
OODLEX_FILEINFO_FLAGSOodleX low level async ioOodleXFileMode

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Init_GetDefaults_Threads
Navigation
 OodleX_Init_GetDefaults_DebugSystems
 OodleX_Shutdown_LogLeaks
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleX_Init_GetDefaults_Threads
{
    OodleX_Init_GetDefaults_Threads_No = 0,
    OodleX_Init_GetDefaults_Threads_Yes = 1,
    OodleX_Init_GetDefaults_Threads_Force32 = 0x40000000
};
Discussion
Should GetDefaults enable any threads?
 
OodleX_Init_GetDefaults_DebugSystemsOodleX Startup and ShutdownOodleX_Shutdown_LogLeaks

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
FAQ: Do new Oodle versions break data compatibility ?
Navigation
 FAQ: What are the Oodle deprecated compressors ?
 Frequently Asked Questions
 FAQ: How does OodleLZ compare with other compressors ?
 Welcome to Oodle
 Change Log

The short answer is NO : Oodle LZ compressed data is intended to be a lifetime supported bitstream format. New versions of Oodle can read all data written by previous versions of Oodle.

If you have old compressed assets, and you want to know can I update to a new version of Oodle (to get a bug fix or performance boost) and still load the old compressed assets - YES , always.

For details, continue reading :

Oodle will always be able to load old compressed data. That is, newer code will be forever backward compatible with old compressed data.

Once in a rare while I rev a format to add new capabilities to the bitstream. When I do that, new compressed data cannot be loaded by older code.

But - my intention is that when possible there is an option to stick with outputing the old format. This is exposed as

OodleConfigValues::m_OodleLZ_BackwardsCompatible_MajorVersion

(the "major" version is the one in the middle ; eg. 2.5.0 has a major version of 5)

For example Oodle 2.3.0 rev'ed the Kraken compressed format; this was the changelog :

Release 2.3.0 - July 14, 2016

  • change : WARNING Kraken data made by 2.3.0 by default cannot be loaded by Oodle 2.2.0 ; set m_OodleLZ_BackwardsCompatible_MajorVersion if you need that. Oodle can always load data made by previous versions, but the reverse is not necessarily true.

So if you have a version of Oodle >= 2.3.0 it will load data that's made by earlier versions.

By default Kraken data made by Oodle >= 2.3.0 cannot be loaded by earlier versions.

If you set OodleConfigValues::m_OodleLZ_BackwardsCompatible_MajorVersion to 2 , then the new modes will be disabled and the newer Oodle can make data compatible with earlier versions.

This could be useful if you have already shipped a game with an older version of Oodle, and you want to deliver new content but don't want to force an update of the decoder, but you do want to take a new version of the encoder (say for a bug fix or whatever), so you can continue to make content that works with the older decoders that customers have.

Obviously when I add new compressors, they will make data that can't be loaded by earlier versions. That is not explicitly called out or handled by the "BackwardsCompatible" system - it's up to you to choose compressors that are supported in the versions you have.

For example Mermaid & Selkie start in Oodle 2.3.0 , so if you use them then earlier versions won't load that data.

The only time I would ever completely break the format, requiring a full recompress of all content, would be if there was a major bug that I couldn't fix any other way. That's never happened and is unlikely, but I definitely advise you to always keep originals of all data just in case.

Special note about Hydra :

If you use Hydra, it can make use of any of the compressors available. When a new compressor is added to Hydra, it could make compressed data that is not compatible with older versions. eg. in Oodle 2.6.0 Leviathan is added to Hydra, which means that Hydra data made using Oodle 2.6.0 cannot be loaded with earlier versions of Oodle. This can be changed by setting m_OodleLZ_BackwardsCompatible_MajorVersion to 5, which will exclude Leviathan from Hydra's menu to ensure it makes data that can be loaded with the previous version.
 

FAQ: What are the Oodle deprecated compressors ?Frequently Asked QuestionsFAQ: How does OodleLZ compare with other compressors ?

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleCore_Plugin_Printf
Navigation
 t_fp_OodleCore_Plugin_WaitJob
 t_fp_OodleCore_Plugin_DisplayAssertion
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC void( OODLE_CALLBACK t_fp_OodleCore_Plugin_Printf )( int verboseLevel,
                const char * file,
                int line,
                const char * fmt,
                . . . );
Discussion
Function pointer to Oodle Core printf
Parameters
verboseLevel  verbosity of the message; 0-2 ; lower = more important
file  C file that sent the message
line  C line that sent the message
fmt  vararg printf format string
Discussion

The logging function installed here must parse varargs like printf.

verboseLevel may be used to omit verbose messages.
 

t_fp_OodleCore_Plugin_WaitJobCore pluginst_fp_OodleCore_Plugin_DisplayAssertion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_CompressionLevel_GetName
Navigation
 OodleLZ_GetSeekEntryPackedPos
 OodleLZ_Compressor_GetName
 Welcome to Oodle
 Change Log
// Function prototype:
const char * OodleLZ_CompressionLevel_GetName( OodleLZ_CompressionLevel compressSelect );
Discussion
Provides a string naming a OodleLZ_CompressionLevel compressSelect
 
OodleLZ_GetSeekEntryPackedPosOodleAPI_LZ_CompressorsOodleLZ_Compressor_GetName

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
Change Log
Navigation
 Welcome to Oodle
 Change Log

Release 2.9.11 - Oct 9, 2023

  • new : Oodle Data and Oodle Network for Apple VisionOS (beta).

  • change : Xbox One legacy XDK support removed. Xbox One GDK/GameCore continues to be suppported.
  • change : Xbox built with GDK March 2023 Update 6
  • change : PS4 built with SDK 10.508
  • change : PS5 built with SDK 7.00
  • change : Switch built with SDK 16.2.3

  • change : Data : Usage warning when decoding to slow-to-read (likely uncached) memory updated to provide more detail.

  • enhancement : Texture : Improved end-to-end latency in multi-threaded encoding, especially for BC1 and BC4/5. (Activity tiling)
  • enhancement : Texture : AVX-512 paths are significantly faster on AMD Zen 4 CPUs. (Avoid memory-destination forms of VPCOMPRESSD)
  • enhancement : Texture : re-designed BC7 mode/partition selection logic that typically gives better quality while simultaneously encoding faster. Especially for diffuse albedo, we've seen RMS error reduced by 2-3% while simultaneously being about 2x faster to encode at "Normal" or "High" effort level.

  • fix : Texture : BC7 RDO fix bug in "preserve extremes" mode where extremal alpha values weren't correctly preserved in certain rare cases. (Add check for mode 7 pbit mismatch in are_indices_permitted check)

  • regression : PS4 Leviathan decode speed about 2-3% slower after compiler update

Release 2.9.10b - Apr 19, 2023

NOTE: re-release due to a buffer overflow issue in Oodle Texture 2.9.10 (Apr 17), please update immediately.

  • new : Data : OodleLZ_GetCompressScratchMemBoundEx which can estimate typical, not just worst-case compression scratch memory bounds.

  • enhancement : Data : Mermaid "Optimal1" (level 5) and higher levels now encode much faster (35-50% encode time reduction is typical), usually with slightly increased compression ratio as well.
  • enhancement : Data : Selkie, Kraken and Leviathan "Optimal1" (level 5) levels have faster encoding (10-25% encode time reduction is typical). "Optimal2" and higher levels slightly faster as well.
  • enhancement : Texture : Speed-ups to RDO activity mask generation, RDO encoding of all formats got slightly faster as a result.

  • fix : Texture : RDO threading changes in 2.9.6 disabled threading of activity calc in most cases, re-tune heuristics.
  • fix : Texture : Fix assertion failure when encoding 16384x16384 pixel BC5 textures from 4x U16 source data.
  • fix : Texture : Fix float input pixels for non-BC6 formats triggering assertions when RDO encoding except for tiny input images.
  • fix : Texture : Fix incorrect RDO encoding of BC6H surfaces with non-multiple-of-4 width or height and at least 16 blocks. Due to the interaction of these two criteria, typical power-of-2 textures with aspect ratios between 1:4 and 4:1 were generally unaffected. The bug typically results in higher visual fidelity and lower compression ratio than indicated by the target lambda, i.e. sub-optimal compression but no objectionable artifacts.
  • fix : Texture : Input pixel format validation is now stricter. Previous versions erroneously accepted some unsupported input formats, e.g. 4x U16 pixel formats for BC1 encoding, then failed with internal errors later.

  • note : Oodle 2.9.10 will be the last release with Xbox One XDK libraries. Xbox One support in upcoming releases will be GDK only.

Release 2.9.9 - Feb 8, 2023

This release focuses on Oodle Data compression speed.

  • new : AArch64 (ARM 64-bit) simulator libraries for tvOS. tvOS distributions now also include the libs/headers packaged as an .xcframework.

  • enhancement : Data : Kraken "Fast" (level 3) now takes typically around 10% less time.
  • enhancement : Data : Kraken "Optimal1" (level 5) and higher levels encode in typically around 7-10% less time. This is independent of speed-ups from encoder scratch memory (see below) which can be in the same ballpark when encoding lots of independent chunks, sometimes much larger.
  • enhancement : Data : Optimal1 and higher levels for all codecs now use provided scratch memory much more extensively and can sometimes avoid separate allocations completely if enough scratch memory is supplied. Especially when encoding many small, independent chunks, reusing allocations in this way can be a massive throughput increase on many-core machines. We've observed speed-ups of around 8.5x when encoding 256k Kraken chunks at Optimal2 level on a 64-core Threadripper. See "About Compression Scratch Memory" in the docs.
  • enhancement : Data : OodleX parallel compress calls set up and reuse encoder scratch memory by default.
  • enhancement : Data : "Optimal1" (level 5) compression on 64-bit ARM typically 10-15% faster independent of other changes noted above. (CTMF improvements)
  • enhancement : Texture : BC7 RDO encoding is slightly faster, 5-8% reduction on typical textures in our test set.

  • fix : Data : fix incorrect Linux 64-bit build settings for some files resulting in missing .note.GNU-stack sections and thus executable stacks.
  • fix : Data : fix OodleX not balancing worker threads across multiple processor groups correctly if number of cores in a processor group wasn't 64. (Incorrect SetThreadGroupAffinity with ~0 affinity mask.)
  • fix : Texture : Fix bug in float->sRGB conversion on x86 targets

Release 2.9.8 - Sep 28, 2022

  • new : Oodle Data for WASM is now out of beta! Beta customers are encouraged to update due to smaller library size and encoder/decoder speed improvements. Please read "About Oodle on WASM" in the docs for integration notes.
  • new : AArch64 (ARM 64-bit) simulator libraries for iOS. iOS distributions now also include the libs/headers packaged as an .xcframework.
  • new : Texture : Support encoding from and decoding to float for BC1-5 and BC7 too, not just BC6H.
  • new : Texture : BC1-3 and BC7 encode from float can now convert linear->sRGB (OodleTex_BCNFlag_LinearToSRGB), decode to float can convert sRGB->linear (OodleTex_BCNDecodeFlag_SRGBToLinear).
  • new : Texture : Added OodleTex_DecodeBCN_Blocks_Ex and OodleTex_DecodeBCN_LinearSurfaces_Ex with "decode flags" argument to enable sRGB->linear encoding.

  • change : iOS static libs now come as pairs of libs, libfoo.a and libfoo.sim.a, with the latter being for simulator. AArch64 (ARM64) targets exist for both device and sim so a single lib is no longer possible.

  • enhancement : Data : Enable "destination is in write-combined memory" usage warnings for Windows/Linux/Mac x64 targets, not just game consoles
  • enhancement : Texture : significantly improved rate-distortion performance of BC7 "preserve extremes" encoding
  • enhancement : Texture : reduced worst-case memory use for BC1, BC3, BC4, BC5, BC7 RDO encoding (index merge heap size limit enforced).

  • fix : Data : fix confusing symbol table entries around address 0 showing up in backtraces on NULL pointer fn calls on some ELF targets (NASM issue)
  • fix : Data : fix memory leak in certain cases for Optimal2+ compression levels introduced in 2.9.6 (rrPool leak when allocation was attempted from scratch arena, but failed and fell back to user allocator instead)
  • fix : Texture : Fix incorrect constant for "prefer wide vectors"/AVX-512 flag in public API (which didn't work correctly as a result)
  • fix : Texture : Fix "preserve extremes" mode for BC7 RDO not preserving pixels correctly in certain rare cases (sometimes not round-tripping mode 7 endpoints in in-loop matrix pass)
  • fix : Setting malloc/free allocator pointers to 0 through the plugin mechanism now installs always-failing dummy allocators instead of crashing on allocation.

Release 2.9.7 - June 30, 2022

  • fix : Network : fix a crash bug in the dictionary builder introduced in 2.9.6 (m_scratch uninitialized)

  • change : Data : OodleX Handle Table size is now growable up to 2048 times what it was before (2^m_num_handles_log2), making it practically unlimited
  • change : Data : DirectStorage example updated for DirectStorage 1.0.2
  • change : Texture : Encoder enables AVX-512 by default on more machine types.

  • enhancement : Texture : Encoder now gives bitwise same results on all supported host platforms (Windows x64, Linux x64/ARM64, Mac x64/ARM64). This will be guaranteed going forward.
  • enhancement : Texture : Significantly decreased memory usage for BC3-5 RDO encoding, slightly decreased for BC1-2.
  • enhancement : Texture : "Preserve extremes" mode (preserves 0/1 in alpha channel exactly) is now supported for BC7. New BCNFlag name is OodleTex_BCNFlag_PreserveExtremes_BC3457, but the old OodleTex_BCNFlag_PreserveExtremes_BC345 will continue to be supported.
  • enhancement : Texture : Encoding on ARM64 hosts is now significantly faster. On M1 Macs in particular, native ARM64 is now around ~2x faster than Rosetta x86_64 emulation, which used to be the faster path.

  • note : Oodle 2.9.7 will be the last Oodle release to support 32-bit iOS, tvOS, MacOS targets or 32-bit x86 Linux. (Windows and Android 32-bit targets, as well as 32-bit ARM Linux, continue to be supported.)

Release 2.9.6 - May 2, 2022

  • fix : Texture : fix an int overflow on textures 2 GB or larger, causing an effective under 2 GB limit for earlier versions, for example on RGBA float textures of 16k x 8k or larger.
  • fix : Texture : fix incorrect assert in BC1 RDO encoder in debug builds when encoding BC1 textures without transparency that had fully black pixels
  • fix : Texture : decreased error in baseline/RDO BC7 encodings for areas with certain singular covariance matrices

  • new : Data : New example showing how to use Oodle Data with DirectStorage on Windows

  • change : Data : improve error messages when trying to decode data that is not an Oodle stream
  • change : Texture : Linux builds now enable AVX2 and higher support (previously disabled). Texture encoding on Linux should be significantly faster.
  • change : Texture : Reduce number of small OodleJobs spawned, especially when compressing small mip levels.

  • enhancement : Texture : BC1 RDO now produces slightly smaller results at same perceptual quality (or equivalently, better quality at the same size). The same improvement applies to RGB portion of BC2 and BC3.
  • enhancement : Texture : AVX-512 support. We've seen up to 30% encode time reduction on AVX-512-supporting machines when encoding BC1, less for other formats. See "AVX-512" in the "Oodle Texture overviews" section in the documentation for usage notes.

Release 2.9.5 - October 13, 2021

  • fix : Texture : fix bug in BC1 RDO encoder incorrectly emitting transparent black blocks in very rare cases
  • fix : Texture : fix bug in BC1 RDO encoder not preserving BC1 transparency correctly in certain rare circumstances

  • new : UWP ARM64 builds now provided

  • change : Texture : BC1-3 decoders have changed, and the encoders now target the new decoders instead of the old ones (which matched the D3D reference). See "BCn decoding" in the "Oodle Texture overviews" section of the docs for rationale.

  • enhancement : Texture : BC1-5 RDO encoding is significantly faster (often 3x or more); BC1 non-RDO encoding up to 2x faster; BC7 RDO encoding typically 1.13x faster.

Release 2.9.4 - September 2, 2021

This is a re-release of 2.9.3 with a change in compiler switches to fix a binary compatibility issue on platforms compiled using MSVC. There are no other changes.

  • fix : Disable FH4 exception handling for link compatibility with VC 2017

Release 2.9.3 - July 26, 2021

  • fix : Data : Fix crash encoding at level Optimal1 on buffers just over block size (eg. 256k+2 bytes) when matches cross block boundary (CTMF)
  • fix : Data : Fix crash when calling OodleLZ_GetCompressScratchMemBound for OodleLZ_Compressor_None introduced in 2.9.1
  • fix : Data : Fix debug lib on 32-bit Linux inadvertently using SSE4.1 in some static initializers prior to CPUID check
  • fix : Data : Fix Oodle decompress erroneously reporting input as corrupted when the compressed data buffer is very close to the top of virtual memory

  • new : Network : OodleNetwork1UDP_State_Compact_ForVersion and OodleNetwork1UDP_State_Uncompact_ForVersion allow you to read/write compacted Network state for old versions (eg. 5 for 2.5.x); compacted binary state changed from v5 to v6

  • change : Linux built with Clang 12
  • change : Linux built with Jcc erratum mitigation (-mbranches-within-32B-boundaries)
  • change : NX built with SDK 11.4.3
  • change : PS4 built with SDK 8.500

  • enhancement : Faster Leviathan encoding at "Normal" and "Optimal" levels
  • enhancement : CPU BC7Prep decode will now use AVX2 256-bit instructions when available, for a speed win of about 25% typically (it's frequently memory bound). You can disable usage of wide vectors when desired via OodleTexRT_BC7PrepDecodeFlags_AvoidWideVectors (see docs).

  • regression : minor regression (5-6%) in encode speed for Mermaid/Selkie on PS4 at some compression levels after update to SDK 8.5 and clang 11

Release 2.9.2 - June 16, 2021

  • fix : Texture : fix crash bug in RDO encoding with Universal Tiling option enabled on some texture sizes

Release 2.9.1 - June 7, 2021

  • fix : Data : Fix incorrect warnings about OodleLZ_Compressor_None being deprecated
  • fix : Texture : fix bug where large textures (eg. 8k x 8k) and high job worker counts (eg. 128) could cause an int overflow leading to a crash or huge alloc. Previous versions can work around this issue by clamping image size or worker count in OodleTex_Plugins_SetJobSystemAndCount.
  • fix : Texture : fix bug in negative signed float values in BC6H RDO encoding producing garbage results.
  • fix : Texture : fix high-error blocks in certain very rare cases when encoding BC1-3 textures without RDO at "High" setting

  • change : MS platforms built with VC 2019 (except Durango-XDK still built with 2017)
  • change : Windows built with JCC mitigation (/QIntel-jcc-erratum) and new FH4 exception handler
  • change : Oodle Texture functions now have an explicit, enforced limit on surface dimensions (OODLETEX_MAX_SURFACE_DIMENSION)
  • change : OodleTex_RMSE_Normalized_BCNAware on BC6/float changed metric, values will not compare to previous versions

  • enhancement : Faster Kraken decompression on Zen
  • enhancement : Faster decompression on Mac M1 ARM64
  • enhancement : Faster Oodle Texture RDO encoding of BC4/BC5 even at highest quality level.
  • enhancement : New entry point OodleTex_EncodeBCN_RDO_Ex that enables several new features like RDO effort levels and Universal Tiling.
  • enhancement : Oodle Texure RDO encoding now has an "effort level" parameter, just like baseline encode. Lower effort levels are significantly faster to encode but reduce quality/compression slightly. Use for fast iteration, higher effort for final builds.
  • enhancement : Oodle Texture Universal Tiling, to share a single RDO encode for linear and platform-specific tiled formats. Minor hit in compression rate, but much better than always RDO encoding in linear layout and much faster than encoding per-platform.
  • enhancement : Can pass OODLETEX_JOBS_DISABLE to Oodle Texture functions to cause them not to spawn any jobs even when a job system is installed. Intended for cases that run many encodes in parallel anyway (e.g. Virtual Texture tiles) where spawning more fine-grained jobs isn't useful.

Release 2.9.0 - March 23, 2021

  • fix : OodleLZ_Compress had an encoder crash bug in the Optimal level encodes on data in sizes just slightly over a 256KB chunk (eg. 262145) with a repeated substring at the very end

  • change : Mac libs and dylibs are now fat binaries with x64 and ARM64
  • change : Tex : Oodle Texture no longer checks for license file
  • change : defining OODLE_IMPORT_LIB / OODLE_IMPORT_DLL is no longer needed; you can link with either type of lib without setting a define
  • change : Oodle public headers no longer define types like U8, SINTa, they are instead OO_U8, OO_SINTa, etc.
  • change : Oodle public headers now require stdint.h which on Windows/MSVC means VC2010 or higher

  • change : Net : OODLE_PLATFORM_HAS_SELECTDICTIONARYANDTRAIN define removed. Call OodleNetwork1_SelectDictionarySupported instead.

  • removed : Core : support for the deprecated LZ compressors is now removed (LZH,LZA,etc.). Only newlz (Kraken,Mermaid,Selkie,Leviathan,Hydra) and LZB16 are supported.
  • removed : Core : OodleLZ_CompressContext_* functions removed; streaming encoders no longer supported
  • removed : Ext : OODLEX_PATH_* public defines removed.
  • removed : Ext : OODLEX_WCHAR_SIZE public define removed.
  • removed : Tex : OodleTex_CheckLicense func removed ; Oodle Texture no longer checks for license files

  • deprecation : OodleConfigValues::m_OodleLZ_Small_Buffer_LZ_Fallback_Size no longer used; newlz compressors no longer ever drop down to LZB16 (default behavior unchanged)

Release 2.8.14 - 2.8.x long term support release version

update March 21, 2021

  • fix : OodleLZ_Compress had an encoder crash bug in the Optimal level encodes on data in sizes just slightly over a 256KB chunk (eg. 262145) with a repeated substring at the very end

  • fix : PS4 decode speed regression fixed

Release 2.8.14 - February 15, 2021

  • enhancement : BC7 encoding is faster ; slightly different encodings at higher speed with similar quality

  • new : Mac ARM64 build now provided ; Mac example exes are fat x64+arm64
  • new : Apple tvOS build now provided

  • deprecation : Mac 32 bit x86 build no longer provided

  • change : Xbox built with Nov 2020 GDK
  • change : PS5 built with SDK 2.0
  • change : PS4 built with SDK 8.0
  • note : NX built with SDK 9.4.1
  • note : Windows built with MSVC 2017

  • regression : PS4 decode speed worse due to clang 10 in SDK 8.0

Release 2.8.13 - November 4, 2020

  • fix : BC1 RDO could compute different results on Intel and AMD processors

  • change : remove use of CRT in MSVC builds to remove -MT/-MD incompatibility

Release 2.8.12 - September 18, 2020

  • change : Unreal Oodle Texture plugin now has support for all consoles
  • change : Unreal Oodle Data plugin better support for soft disable and per-platform disable

  • change : Texture full license files never stop working even if they are expired; eval licenses add 30 day grace period

  • enhancement : Fix some regressions from 2.6.3 in OodleLZ encode & decode speed, particularly with Mermaid and Selkie on tiny buffers (tested on 1k and 4k)
  • enhancement : clang10 Linux build with Jcc erratum mitigation workaround, and vc 2019, optional builds (gcc 47 and vc 2017 primary builds still available)

Release 2.8.11 - rerelease - April 11, 2022

  • license check removed

  • Call to rsqrt removed. Encoding will match previous behavior on Intel. Use of reciprocal sqrt instruction caused texture encodings to be different on Intel and AMD processors.

Release 2.8.11 - August 20, 2020

  • change : OodleTex_BC1_WithTransparency and OodleTex_BC2 now also support RDO ; OodleTex_BC1_WithTransparency can be used to preserve opaque alpha in the BC1 encoding.

  • fix : Unreal integration for Oodle Texture was incorrectly using OodleTex_BC1 which has undefined alpha value; Unreal requires OodleTex_BC1_WithTransparency to output opaque alpha

  • enhancement : OodleTex_BC1_WithTransparency non-RDO encode faster (was slower than BC1, now roughly same speed)
  • enhancement : BC1-BC3 RDO quality very slightly better

Release 2.8.10 - August 4, 2020

  • new : OodleTex_PixelFormat_GetName and OodleTex_BC_GetName helpers

  • change : OodleTex_EncodeBCN_RDO and OodleTex_EncodeBCN_LinearSurfaces now accept more than 1 input surface with layout = NULL, so all mips/slices can be encoded in one call
  • change : otexdds example now does all mips/volumes as a single Encode call for better parallelism
  • change : default OodlePlugins_SetPrintf log plugin in MSVC platforms now uses stdio, creating a ucrt VC 2015+ dependency (previously all other platforms used stdio but MSVC platforms did not)
  • change : (Windows only) when license is expired, it shows a message box and allows you to keep running rather than just failing

  • fix : license file date compare could incorrectly report an expired license when it wasn't
  • fix : potential race causing a crash in example_jobify_linuxtbb.inl (same as example_jobify_win32tp.inl fix in last release)

Release 2.8.9 - July 25, 2020

  • new : Oodle Texture integration for UE4 (TextureFormatOodle for UE4.25.1)

  • enhancement : Oodle Texture RDO encode much faster on large images

  • change : Oodle Windows static lib build was using static lib CRT, changed to DLL CRT.
  • change : OODLE_WORKERS_COUNT_ALL_HYPER_CORES can be used with OodleXInitOptions m_OodleInit_Workers_Count
  • change : iOS libs incorrectly had the version number in the name (Oodle static libs do not have version number in the name, dynamic libs do)

  • fix : Oodle Network training could fail on input larger than 2 GB (32 bit count overflow)
  • fix : potential race causing a crash in example_jobify_win32tp.inl in dep link lifetime

Release 2.8.8 - July 7, 2020

  • new : OodleTexRT_PS5GPU_BC7Prep_DecodeMulti to decode multiple BC7Prep chunks at once with reduced memory use and GPU synchronization overhead
  • new : GPU-side BC7Prep decoding now supported on Xbox Series X, Xbox One and PS4

  • enhancement : CPU-side BC7Prep decoding is significantly faster for large textures or when decoding to write-combined memory, and requires less scratch memory
  • enhancement : PS5 GPU BC7Prep decode is 2x-3x faster, uses less GPU memory bandwidth, and significantly less scratch memory

  • change : rename OodleTex_PixelFormat_3_F32_RGBA to OodleTex_PixelFormat_3_F32_RGB
  • change : PS5 GPU BC7Prep no longer implicitly performs any cache invalidations; this is now up to the app.

  • deprecation : OodleTexRT_BC7Prep_PS5GPU_Init is deprecated, call OodleTexRT_PS5GPU_Init instead
  • deprecation : OodleTexRT_BC7Prep_PS5GPU_MinDecodeScratchSize is deprecated, call OodleTexRT_PS5GPU_BC7Prep_MinDecodeScratchSize instead
  • deprecation : OodleTexRT_BC7Prep_PS5GPU_Decode is deprecated, use OodleTexRT_PS5GPU_BC7Prep_DecodeMulti instead

  • fix : Xbox import lib names for DLLs had version number in the lib name; remove to match Windows names. Version number is in the DLL but not the import lib name.
  • fix : 1 and 2 channel U16 promotion to 4 channel was not filling the implicit opaque alpha correctly, affected BC4 and BC5 decodes to 4 channel formats

Release 2.8.7 - June 8, 2020

  • new : Oodle Texture! Oodle Texture is a new product separate from Oodle Data. Oodle Texture dramatically reduces the size of block-compressed BC1-BC7 textures.

  • change : executables provided with SDK are now in bin dir
  • change : oodle2base.h header with basic types now shared by Oodle Data, Net, and Texture
  • change : example_jobify now provides a plug-in job system that can be used with Oodle Data, Net, and Texture
  • change : t_fp_Oodle_Job common data type for Jobify functions in Oodle Data, Net, and Texture (replaces t_fp_OodleCore_Plugin_Job)
  • change : OODLE_JOB_MAX_DEPENDENCIES common define for Jobify in Oodle Data, Net, and Texture (replaces OODLECORE_PLUGIN_JOB_MAX_DEPENDENCIES)
  • change : _SALSA_ define no longer needed for PS5 Oodle header
  • change : remove OodleLZ_CompressionLevel_Count ; the correct range is [Min,Max] because of negative levels

Release 2.8.6 - May 9, 2020

  • new: OodlePlugins_SetJobSystemAndCount replaces OodlePlugins_SetJobSystem with desired parallelism argument

  • enhancement : Leviathan decodes 5-10% faster on modern x86/64 platforms.

  • change : Oodle build for Windows now compiled with MSVC 2017 and uses ucrt
  • change : Oodle build for Mustard now compiled with MSVC 2017
  • change : fix Mac and Linux debug & dead stripping

  • deprecation : remove async Cancels from the public API (OodleX_CancelOrWait_AndDelete, OodleX_WaitCancelAllPending, OodleXIOQ_Pause, OodleXIOQ_WaitCancelAllPending)
  • deprecation : m_OodleLZ_Desired_Parallel_BranchFactor removed from OodleXConfigValues, now automatically scaled to worker count
  • deprecation : OodlePlugins_SetJobSystem is deprecated. Prefer OodlePlugins_SetJobSystemAndCount.
  • deprecation : OodleXPriority levels are no longer exposed to the client. Client work should be OodleXPriority_Normal only. Prevents a potential deadlock bug mixing high priority client work with Core Jobs.

Release 2.8.5 - March 10, 2020

  • fix : iOS build could assert "OodleAssert, 32 >= 48" in debug builds; this was also a bug in release builds on iOS that could cause stack variable corruption; do not use sendQuantumCRCs on iOS before this version!

  • new: initial Mustard release version
  • new: Xbox One GDK release version
  • change : Stadia build variant

Release 2.8.4 - October 31, 2019

  • fix : OodleNetwork1UDP_Encode could read one byte past the end of the input buffer, causing an access violation. The extra byte read was not used in coding, there's no problem with the compressed packets made. Allocating the input buffer with padding is an acceptable workaround for older versions.

Release 2.8.3 - October 4, 2019

  • new: initial Salsa release version

  • change : Stadia build SO extracts debug info to .debug file and uses add-gnu-debuglink to link to it
  • change : disable debug info in Linux ARM64 build because of TLS debug relocation bug in clang

Release 2.8.2 - September 19, 2019

  • fix : Rare non-critical issue : "WARNING: Too many indices for decoder scratch!" could be logged by the encoder in Leviathan Level 9. Valid compressed data was still made, but in some cases it was suboptimal.
  • fix : Rare crash bug in Optimal level encoders on large buffers (LRM degeneracy jumpInShift 32) (since 2.8.1)
  • fix : fix some spurious valgrind detections in Oodle (there were no bugs)

Release 2.8.1 - July 7, 2019

  • new: Oodle Core API : OodleLZ_GetFirstChunkCompressor preferred name of old API OodleLZ_GetChunkCompressor
  • new: Oodle Core API : OodleLZ_GetAllChunksCompressor should be used when chunks may have heterogeneous compressors (eg. with Hydra)
  • new: Oodle Ext API : OodleX_CorePlugin_RunJob and OodleX_CorePlugin_WaitJob
  • new: Oodle Ext API : OodleX_GetNumWorkerThreads

  • enhancement : reduce OodleLZ_GetCompressScratchMemBound, particularly on small buffers and small hash table options, and more so for Mermaid/Selkie
  • enhancement : OodleLZDecoder_MemorySizeNeeded was increased in version 2.8.0 to allow for apparent Mermaid/Selkie data possibly being Hydra ; it is now reduced back down and OodleLZ_GetAllChunksCompressor should be used to detect Hydra data.
  • enhancement : Optimal level encoders 5-10% faster on large buffers (LRM Bloom filter)

  • change : Hydra data now needs to be explicitly identified when used as the OodleLZ_Compressor type in functions like OodleLZDecoder_MemorySizeNeeded (or use OodleLZ_GetAllChunksCompressor)
  • change : built with Switch SDK 7.4.0 (clang 7.0.1)
  • change : built with PS4 SDK 6.5 (clang 7.0.1)

  • fix : Switch : fix intermittent problem with debugging in the VSI in apps that use Oodle; Oodle linkage confused the debugger.
  • fix : OodleLZ_GetCompressedStepForRawStep was incorrectly casting the return value to 32 bit, making it wrong for steps over 2 GB
  • fix : fix rare read out of array bounds in Kraken & Leviathan encoder (multiarrays non-indexed splitter), would not be an access violation or compression failure, but could cause unexpected behavior and non-deterministic encoding.

  • deprecation : OodleLZ_GetChunkCompressor will be removed in the future; use OodleLZ_GetAllChunksCompressor or OodleLZ_GetFirstChunkCompressor
  • deprecation : OodleXMallocCall family of functions were incorrectly public, removed

Release 2.8.0 - April 9, 2019

  • new : OodleCore_Plugins_SetJobSystem for Job plugin system to thread work in Oodle Core using a user-provided worker thread system (see Oodle_About_Job_Threading_Plugins)
  • new : OodleLZ_GetCompressScratchMemBound to query how much scratch is needed for OodleLZ_Compress to avoid additional allocations
  • new : example_lz_noallocs : example demonstrating Oodle compression & decompression with no allocations done by Oodle

  • enhancement : Optimal level encoders are faster, particularly Kraken, and even more so with Jobify threading (but also faster single threaded)
  • enhancement : Leviathan fast levels (SuperFast-Normal) encode much faster with a small decrease in compression ratio

  • change : API : WARNING CompressOptions struct changed. Old fields have not moved, new fields have been added at the end of the struct. Zero initializing the new fields means "use default".
  • change : CompressOptions:jobify option to control threading of the OodleLZ_Compress encoder in Oodle Core (see Oodle_About_Job_Threading_Plugins)
  • change : CompressOptions:farMatchMinLen option to allow decode platform cache targeting of the encoded stream
  • change : CompressOptions:spaceSpeedTradeoffBytes value of zero now means "use default" (256) to make it consistent with all other Options taking zero for "default". Use negative to get zero.
  • change : docs are now html instead of chm
  • change : Android x86_64 build
  • change : add a Mermaid HyperFast4 level (previously HF4 was the same as HF3 in Mermaid)
  • change : API : OodleLZ_GetDecodeBufferSize, OodleLZ_GetCompressedBufferSizeNeeded and OodleLZ_GetInPlaceDecodeBufferSize : take compressor argument to return smaller padding for the new codecs.
  • change : API : OodleLZ_GetChunkCompressor : take compressed size argument to ensure it doesn't read past end
  • change : Kraken Optimal1 level gets a bit less compression in trade off for much faster encoding

  • fix : WARNING! Nasty destructive bug in the example code : examples calling "make_example_input" were stomping the input file. This was not in the Oodle libs, just the example code.
  • fix : Leviathan no longer uses the allocator if sufficient scratch mem is passed in to OodleLZ_Compress
  • fix : bug in Leviathan scratch accounting could rarely make it refuse to encode some chunks (valid compressed data would still be made, but it might be suboptimal)
  • fix : Mermaid/Selkie in-place decompression could fail ("in place" decoding is when the input compressed and output decompressed buffers overlap)
  • fix : OodleX worker threads restored to LIFO wakeup order after being incorrectly FIFO for a few versions (improves cache coherence)

  • deprecation : OodleXLZ_ReadCompressWrite_Async API removed
  • deprecation : CompressOptions previously deprecated maxHuffmansPerChunk now unused
  • deprecation : CompressOptions verbosity deprecated; wasn't used by the new compressors anyway

Release 2.7.6 - December 18, 2018

  • enhancement : Linux ARM64 : Mermaid/Selkie decode faster

  • change : Android libs now in abi subdirectories
  • change : added new usage warning that verifies the compressed buffer passed in to OodleLZ_Compress is at least OodleLZ_GetCompressedBufferSizeNeeded bytes long; can be disabled with Oodle_SetUsageWarnings

  • fix : Fix Leviathan at Optimal levels using a large amount of stack memory (was 512k, now fits in 64k)
  • fix : Fix the Unreal .build.cs rules for the Oodle integration on non-Windows platforms, for Unreal 4.20
  • fix : Fix the Unreal data integration with bAsync compression turned on there could be a crash due to a race
  • fix : Fix Oodle Network lib name on Switch ; was "oo2corenet" should be "oo2netswitch"
  • fix : Fix a bug in the Mermaid/Selkie decode on Switch that could cause a fuzz safety failure (overrun on corrupt data)

Release 2.7.5 - October 31, 2018

  • new : OodleLZ_Compressor_RespectsDictionarySize property query tells if a compressor obeys OodleLZ_CompressOptions:dictionarySize (all the new codecs do, some old ones do not)
  • new : ozip -b (benchmark) command line argument, similar to zstd -b

  • enhancement : Linux ARM64 : enable ASM optimized kernels in build

  • change : clarify that seek chunk boundaries are relative to dictionary base, not current raw buffer
  • change : old compressor LZH was doing archaic RLE mode at level 1 (superfast), now removed
  • change : Oodle Windows SDK import libs back in the "lib" folder (temporarily was "import_lib")

  • fix : Fix bug with seekChunkReset on very large buffers (over 1 GB)
  • fix : OodleLZ_CompressOptions_Validate was not making seekChunkLen power of 2, which it must be
  • fix : change OODLELZ_SEEKCHUNKLEN_MAX from 1 GB to half GB. (recommend using OodleLZ_MakeSeekChunkLen)

Release 2.7.4 - October 25, 2018

  • fix : Fix bug in Leviathan Optimal5 (level 9) encoder reading uninitialized memory. The encoder did not crash and always made valid data, but it was not deterministic.

Release 2.7.3 - October 10, 2018

  • enhancement : Mermaid, Kraken & Leviathan decompress faster in 32-bit by about 5% on x86, even more on ARM.

  • change : Oodle windows SDK now ships DLLs and static libs. Libs are in "import_lib" and "static_lib". (dir names changed in 2.7.5). To use the static lib define OODLE_IMPORT_LIB.

  • fix : Fix bug that could cause the Kraken & Leviathan encoder to crash at level 8 and 9.

Release 2.7.2 - September 10, 2018

  • fix : Fix linkage in 32-bit iOS build

Release 2.7.1 SDK Update - August 13, 2018

  • change : Switch SDK updated to 5.5
  • change : PS4 SDK updated to 5.5

Release 2.7.1 - August 8, 2018

  • fix : Fix a fuzz safety failure. Decoding corrupt or attack data with Mermaid or Selkie could cause a read access violation.

Release 2.7.0 - August 6, 2018

  • change : Oodle Network is now a separate SDK and lib from Oodle Data compression (Core & Ext).

  • new : OodleLZ_CompressOptions::profile option added (replaces "unused")
  • new : example_lz / lz_test_13 shows how static dictionaries can be used with Oodle LZ via memcpy

  • enhancement : Mermaid, Kraken & Leviathan decompress faster by 5-10%

  • change : OodleHuffman public APIs removed
  • change : OodlePlugins_ API set renamed to OodleCore_Plugins_ and OodleNet_Plugins_ eg. OodlePlugins_SetPrintf -> OodleCore_Plugins_SetPrintf and OodleNet_Plugins_SetPrintf added
  • change : PS3 and Xbox 360 builds removed
  • change : Oodle Mac SDK now requires min OSX version 10.9 with libc++ instead of libstdc++ (note using the dylib prevents most of these problems; it's recommended to use the shared libs instead of the static libs if possible)

  • deprecation : begin removal of archaic option; rename OodleLZ_CompressOptions::maxHuffmansPerChunk -> deprecated_maxHuffmansPerChunk

Release 2.6.3 Update 1 - July 3, 2018

  • fix : iOS lib didn't include bitcode correctly

Release 2.6.3 - June 6, 2018

  • new : new faster "HyperFast" compression levels; OodleLZ_CompressionLevel_HyperFast1-4 provide super fast encoding with lower compression ratios for real-time encoding needs. Available in Kraken, Mermaid & Selkie.
  • new : example_lz_chart is now provided as a pre-built executable for evaluators on desktop platforms
  • new : ozip executable provided on desktop platforms which acts like gzip; can be used for pipe or file compression.

  • fix : Mermaid Normal encoder wasn't doing dictionary preload; this made compression ratio worse than it should be in parallel encodes (a tiny bit).
  • fix : OodleXLZ_Compress_Async was running wide even when OodleXAsyncSelect_Wide flag was not set. Now uses only one thread if Wide is not set.
  • fix : Compression of buffers larger than 1 GB, with Options enabled for seekChunkReset with seekChunkLen of 0.5 GB or less could create invalid compressed data which would fail to decode or not respect the requested seek chunking

Release 2.6.2 - April 30, 2018

  • enhancement : improve parallelism of the OodleX Async wide compression

  • fix : Some SIMD code could try to run the SSE4 variant even on CPU's that don't have SSE4, causing a crash. (Affects x86/x64 Windows, Linux & Mac)
  • fix : XBox One libs failed BinScope due to NASM assembly objects setting wrong MASM version number

Release 2.6.1 - April 4, 2018

  • enhancement : Huffman encoders slightly faster. Fast levels of Mermaid, Kraken, etc. about 1% faster to encode.

  • fix : setting OodleLZ_CompressOptions::maxLocalDictionarySize above 2^26 could cause a crash in the optimal encoders due to S32 wrapping to negative
  • fix : OodleLZ in backward compatible mode for version < 6 would not compress memset chunks correctly (would send them uncompressed); they did not fail to encode or decode, it was just inefficient
  • fix : Selkie wasn't sending memset chunks; now does
  • fix : Oodle Network could assert if only one packet was given to SelectDictionaryFromPackets. This bug affected the debug lib only.

  • deprecation : Optimal encode levels of old codecs LZNIB, LZBLW and LZA removed. Optimal encodes with those codecs now use Normal level encoder.

Release 2.6.0 - Feb 27, 2018

  • new : new compressor Leviathan !!

  • new : new API Oodle_SetUsageWarnings ; usage warnings are on by default
  • new : OodlePlugin_Printf_Verbose added; can be installed in Oodle Core via OodlePlugins_SetPrintf

  • enhancement : Kraken & Mermaid optimal levels now get more compression (chunk adaptation & TLL parse)
  • enhancement : Mermaid optimal levels now get more compression on some files (costing of entropy offsets)
  • enhancement : Kraken & Mermaid generally achieve better space-speed with the >= 2.6.0 bit stream changes (smaller sizes and/or faster to decode)
  • enhancement : Kraken, Mermaid & Selkie fast levels are now much faster to encode! (especially SuperFast and VeryFast)

  • change : WARNING Kraken & Mermaid data made by 2.6.0 by default cannot be loaded by Oodle 2.5 and earlier ; set m_OodleLZ_BackwardsCompatible_MajorVersion if you need that. Oodle can always load data made by previous versions, but the reverse is not necessarily true.

  • change : WARNING Oodle Network Compact/Uncompact is not compatible between before 2.6.0 and after 2.6.0

  • change : OodleLZ_Small_Buffer_LZ_Fallback_Size default value changed to 0 so it never happens unless the client modifies this value to something larger.
  • change : example_lz : added lz_test_12 showing in place decoding
  • change : Hydra can select Leviathan. Hydra-compressed data cannot be loaded by Oodle 2.5 and earlier because they don't contain Leviathan. To use Hydra to make data compatible with previous versions, set m_OodleLZ_BackwardsCompatible_MajorVersion.
  • change : OodleLZ_Compress API change , added scratch memory arguments so client can pass in pre-allocated memory to eliminate internal Oodle alloc calls. (non-optimal levels only in the new compressor family; optimal levels & old compressors will still do allocations even if scratch is provided)
  • change : fix name of enums to match OodleX convention : OodleLog_VerboseLevel renamed to OodleXLog_VerboseLevel , OodleLog_StateFlags renamed to OodleXLog_StateFlags
  • change : secret level 8 (OodleLZ_CompressionLevel_Optimal4) is now public ; no longer modifies compress options or changes space-speed tradeoff target
  • change : OodleLZ_CompressionLevel_Optimal4 Mermaid & Kraken higher compression optimal parse
  • change : Oodle for Switch is now built with SDK 3.4
  • change : OodleHuffman_EncodeArrayU8 takes scratch memory argument to avoid internal allocation
  • change : OodleXLZ_Decompress_ThreadPhased_Narrow_Async takes scratch memory argument to avoid internal allocation
  • change : OodleLZ_Decompress API : default argument value of OodleLZ_FuzzSafe_No removed; recommend passing OodleLZ_FuzzSafe_Yes
  • change : LZH and LZHLW removed from FuzzSafe set. If you were previously decoding them using FuzzSafe_Yes they will now fail.
  • change : OodleXLZ_ReadAndDecompress_Stream_Async API : added decBufBase argument
  • change : OodleLZ_CompressOptions_GetDefault at level 8 was returning a lower spaceSpeedTradeoffBytes ; don't do that anymore, the default is always 256 ; decode speed is mostly independent of compression level.

  • fix : Mermaid/Selkie : some of the non-optimal encode levels were not respecting the dictionarySize option to limit the match window
  • fix : Kraken/Mermaid/Selkie didn't support CompressionLevel_None correctly. They now pass through data uncompressed at that level.
  • fix : Kraken/Mermaid/Selkie decoders had some potential fuzz safety violations (long matches at end of buffer)
  • fix : OodleHuffman_DecodeArrayU8 wasn't doing CPU detection; it was slower than it should be if you didn't call other Oodle functions
  • fix : the OodleXLZ_Compress_WriteOOZ family of functions took seekChunkLen as both as a function argument and via the OodleLZ_CompressOptions , creating a potential for mismatch ; the function argument is now removed.
  • fix : Mermaid encoder was not putting backwards compatible streams correctly for major version 3 (2.3) compatibility (was always making streams for version >= 2.4.0)
  • fix : Mermaid/Selkie encoder could overrun compressed buffer beyond OodleLZ_GetCompressedBufferSizeNeeded on some blocks. In parallel encoding mode this could cause corrupted compressed streams.
  • fix : OodleXLZ_Decompress_ThreadPhased_Narrow_Async had a race in its finalization that could cause a read from freed memory
  • fix : OodleLZDecoder_DecodeSome was incorrectly returning false when compAvail was too small to parse the first header (typically < 16 bytes); now returns true with no progress
  • fix : OodleLZ_Decompress could return -1 for failure; it should always return 0 for failure (OODLELZ_FAILED); check <= 0 in previous versions

  • warning : DEPRECATION : Oodle compressors that predate Kraken are being gradually deprecated. (eg. LZH, LZNA, etc.) You should always be encoding with compressors from the new Kraken family (Kraken, Mermaid, Selkie, Leviathan, Hydra). Old decompressors will be supported for the forseeable future, so old data will still be loaded but the encoders will be removed in some future version. Attempts to encode with outdated compressors is now logged as an Oodle Usage Warning. This can be disabled with Oodle_SetUsageWarnings. The OodleLZ_Compressor enums for the old codecs are now hidden; you must define OODLE_ALLOW_DEPRECATED_COMPRESSORS before including oodle2.h to get them.

  • obsolescence : EASTL is no longer used
  • deprecation : old OodleLZ_CompressionLevel_RLE removed
  • deprecation : OodleX OOZ functions removed (OodleXLZ_Compress_WriteOOZ_Async, etc)
  • deprecation : OodleXLZ_Decompress_ThreadPhased_Wide_Async removed

Release 2.5.5 - August 22, 2017

  • fix : A bug in the Kraken & Mermaid encoders was introduced in Oodle 2.5.0 ; any data encoded with Kraken or Mermaid from Oodle 2.5.0 - 2.5.4 should be checked for correct decompression and re-encoded if needed. Those versions may have made data which will fail to decode, or might claim to succeed decoding but produce bad bytes. The cause was that huff streams in the encoder could collide and overwrite each other without detection. This was most likely in whole-huff chunks at 128k boundaries, on trinary-aligned data (such as RGB BMP data). This was more likely with m_OodleLZ_BackwardsCompatible_MajorVersion <= 2 but could happen in other cases.

Release 2.5.4 - August 8, 2017

  • new : Windows UWP build added

  • change : Oodle PS4 now built with SDK 4.500

  • fix : omit-frame-pointer was set on too many builds; it is now only used in x86 (32-bit) builds

Release 2.5.3 - June 14, 2017

  • fix : remove alloca use in rrHuffman that could cause crash on threads with small stack size on Linux

Release 2.5.2 - May 25, 2017

  • fix : Kraken decoder had a bug that caused it to incorrectly return failure on valid compressed chunks (due to the header corruption check not counting the scratch space needed with correct accounting of the thread phasing header) - NOTE this change means OodleLZDecoder_MemorySizeNeeded is slightly larger now
  • fix : fix handle leak in OodleXLZ_Compress_WriteOOZFile_AsyncAndWait when file open failed

Release 2.5.1 - May 13, 2017

  • enhancement : Mermaid & Selkie encoding improved (faster and better ratio) on buffers <= 64k bytes long

  • change : Oodle Mac OSX build now includes static libs and dynamic libs.
  • change : Xbox One build is made with March 2017 XDK , still VC 2012 (but VC 2015 compatible)

  • deprecation : LZB16 and LZBLW Optimal level encoders are gone. LZB with CompressionLevel >= Optimal1 now drops down to Normal. Use Selkie instead.
  • obsolescence : libdivsufsort is no longer used

Release 2.5.0 - April 20, 2017

  • new : OodleHuffman_EncodeArrayU8 and OodleHuffman_DecodeArrayU8 reintroduced as public APIs. (not compatible with old public Huffman API)

  • enhancement : Kraken, Mermaid & Selkie - the non-optimal encode levels (SuperFast - Normal) get more compression and encode faster
  • enhancement : Kraken, Mermaid & Selkie - faster to decode, especially on ARM (Android, iOS, Switch)
  • enhancement : Hydra encoding now works in levels below Optimal

  • change : NX now uses SDK 3.0

  • fix : Kraken, Mermaid & Selkie - SuperFast encode level had bugs causing it to get much worse compression than necessary
  • fix : LZB16 encoder at the Optimal1 level could crash due to uninitialized memory

Release 2.4.3 - February 14, 2017

  • fix : Mermaid & Selkie encoders could cause an access violation reading past the end of the raw buffer if the length was just over 64k mod 128k
  • fix : Some compressors failed to decode files that were (128k+1)mod 256k bytes long due to bug in fuzz safety checks
  • fix : Selkie encoder was incorrectly choosing offset compression in some cases, causing slower decodes

Release 2.4.2 - January 25, 2017

  • new : Nintendo NX (Switch) support (SDK 0.17.13 and 1.0.0)
  • new : example_lz_outputchunking : Example demonstrating cutting OodleLZ compressed output into fixed size chunks

  • enhancement : Kraken, Mermaid & Selkie now support "in-place" decoding ; OodleLZ_Compressor_CanDecodeInPlace now returns true for all OodleLZ decoders

  • fix : a bug could occur in Oodle Ext 32-bit builds on systems with user memory in the top 2 GB
  • fix : rename OodleLog_Printf macro to OodleXLog_Printf to match OodleX lib naming convention
  • fix : LZB16 decoder crashed on 32-bit ARM Android. LZB16 was also used as the fallback for tiny buffers in Kraken/Mermaid/Selkie/Hydra.

Release 2.4.1 - October 3, 2016

  • new : OodleX_ReleaseThreadTLS function to release OodleX TLS resources in the rare usage that you churn through a huge number of thread creates and destroys

  • change : Clean up the behavior of OodleLZ_GetCompressedStepForRawStep when it is not given the entire compressed data; now returns -1 for error and 0 for not enough data.

  • fix : Fix race in the OodleThinSemaphore used in the example code via OodleX_Semaphore_Post; this was only used by example_lz_threadphased , so should not affect any production code
  • fix : Out of memory default handler would just infinite busy-loop on fgetc on non-Windows platforms
  • fix : Android distribution didn't contain AArch64 libs

Release 2.4.0 - August 22, 2016

  • new : Hydra ; automatically selects Kraken/Mermaid/Selkie
  • new : example_lz lz_test_11 demonstrates sliding window with Kraken using memcpy
  • new : Mermaid+ ; slightly higher compression, slightly lower speed ; control Mermaid vs Mermaid+ using spaceSpeedTradeoffBytes

  • enhancement : LZNA level 8 (Optimal 3) tries several min match lengths

  • fix : Selkie & Mermaid encoders could overrun the compressed buffer when data expanded
  • fix : fix Mac dylib having a different name for its self-reference
  • fix : fix Linux link error with gold on the Oodle static lib
  • fix : fix iOS version min lowered to 7.0 (was 8.1)
  • fix : fix race in some low level thread primitives caused by the threadprofiler. Showed up as ThreadPhased decodes hanging.

Release 2.3.0 - July 14, 2016

  • new : Mermaid ! - Mermaid is a new super-fast-to-decode compressor with good compression ratios (comparable to ZLib)
  • new : Selkie ! - Selkie is the fastest-decoding compressor, with low compression ratios (comparable to LZ4)
  • new : new compressor property query OodleLZ_Compressor_CanDecodeFuzzSafe
  • new : OodleConfigValues for core-only compressor config
  • new : OodleConfigValues::m_OodleLZ_BackwardsCompatible_MajorVersion - set to 2 to make Kraken data that can be loaded by Oodle version 2.2.0
  • new : lz_test_10 in example_lz demonstrates decoding quanta from a finite io buffer

  • enhancement : Kraken is even faster to decode (10-20%) on all platforms
  • enhancement : Kraken significantly faster to decode on ARM

  • change : WARNING Kraken data made by 2.3.0 by default cannot be loaded by Oodle 2.2.0 ; set m_OodleLZ_BackwardsCompatible_MajorVersion if you need that. Oodle can always load data made by previous versions, but the reverse is not necessarily true.
  • change : OodleLZ Decompress functions args changed. New OodleLZ_FuzzSafe argument. Removed the allowed_compressor_mask.
  • change : OodleLZ_CompressOptions dictionarySize option added to limit offsets
  • change : OodleLZ_CompressOptions offsetShift option for old LZH encoder removed (now always 0)
  • change : LZB16 decoder is now fuzz safe. Very slightly slower to decode.

  • fix : LZB16 Optimal1 level could crash in encode of files > 2GB
  • fix : OodleLZ_GetInPlaceDecodeBufferSize was too big on tiny buffers (it's now never bigger than complen + rawlen)
  • fix : iOS is shipped as static libs with extension .a

Release 2.2.0 - May 11, 2016

  • new : threaded Kraken decoder! See OodleLZ_About_ThreadPhasedDecode
  • new : example_lz_threadphased contains a client-side implementation of a ThreadPhased decoder
  • new : Kraken Optimal3 (Level 7) mode with higher compression ratio

  • enhancement : Kraken optimized for ARM processors
  • enhancement : Kraken optimal parse levels improved, about 1% more compression

  • change : XBox1 now built with VS 2012, but Core lib is compatible with VS 2015 (Ext will work in VC 2012 but not in VC 2015)
  • change : Windows SDK redist64 dir removed; 64-bit DLL's are just in "redist"
  • change : examples now include oodle2.h with a relative path so they compile out of the box
  • change : Oodle2 Core lib default plugins no longer use the CRT on Microsoft platforms. See OodleAPI_OodleCore_Plugins
  • change : minor version bumped for API incompatibilities
  • change : OodleLZ_Decode_ThreadPhase argument added to OodleLZ_Decompress and OodleLZ_DecodeSome for 2-threaded Kraken decode. To retain previous behavior, add OodleLZ_Decode_Unthreaded to your call (or use default argument).
  • change : allowed_compressor_mask argument added to OodleLZ_Decompress and OodleLZ_DecodeSome for fuzz safety. To retain previous behavior, add OODLELZ_ALLOWED_COMPRESSOR_ALL to your call (or use default argument).

  • removed : Oodle Huffman public APIs removed

Release 2.1.5 - April 26, 2016

  • new : Kraken !! An amazing new compressor with high compression and fast decode speed!

  • change : example_packet : remove some rarely used options
  • change : PS4 library now built with SDK 3.0
  • change : Xbox One library now built with March 2016 QFE1 , VS 2015

  • fix : LZB16 and LZBLW decoders could over-read compressed data if the last quantum was memset
  • fix : example_packet : better randomization of subsets and limits in TestOodleNetwork_SelectDictionaryAndTrain

  • removed : OodleLZ_GetZipLikeCompressionSettings is gone
  • deprecation : LZHLW and LZNIB and LZBLW compressors are now deprecated, use Kraken instead

Release 2.1.4 - March 20, 2016

  • new : example_lz lz_test_9 demonstrating block composability rules

  • enhancement : LZNA Normal parse compression improved
  • enhancement : LZNA wasn't using SSE on Linux or Mac ; it now does, which makes those platforms much faster

  • change : Oodle Logs at verbosity level 2 are now compiled out of release builds
  • change : example_lz_chart compressor set changed
  • change : OodlePlugins_SetAssertion can now be called with a NULL function pointer argument as an easy way to disable all assert handling
  • change : example_lz test_8 shows how to use the WithContext calls for independent buffers
  • change : OodleNetwork1UDP_State_Uncompact return bool to indicate success or failure on bad data

  • fix : OodleLZ_CompressWithContext with NULL CompressOptions crashed
  • fix : LZBLW Normal level encoder would crash when called with dictionary preload (or large buffers)
  • fix : OodleNetwork1UDP fix fuzz safety for split large packets
  • fix : OodleNetwork1UDP fix fuzz safety prevent reading past end of dictionary
  • fix : OodleNetwork1UDP removed all used of release-asserts, now returns failure in those cases
  • fix : OodleNetwork1UDP properly document OODLENETWORK1_DECOMP_BUF_OVERREAD_LEN

Release 2.1.3 - Feb 1, 2016

  • enhancement : OodleNetwork1UDP_Decode faster
  • fix : OodleNetwork1UDP_Decode fuzz safety
  • fix : OodleNetwork1UDP - fix possible bug in large split packet handling; now properly ensures that complen <= rawlen, and complen == rawlen always means a pass-through uncompressed packet
  • deprecation : OodleLZ_Compressor_CanSPUDecompress function removed

Release 2.1.2 - Jan 16, 2016

  • new : BitKnit ! BitKnit is a new LZ compressor with high compression and good speed, between LZNA and LZHLW
  • enhancement : Most LZ encoders are faster
  • enhancement : LZNA Fast & Normal levels get more compression

Release 2.1.1 - Nov 25, 2015

  • new : example_lz_chart make a neat text chart of the Oodle compressors & levels to test performance
  • new : example_network_client simple Oodle Network client example
  • fix : don't store OODLE_HEADER_VERSION in the example_packet data header; use OODLE2_VERSION_MAJOR instead
  • enhancement : most decoders faster by around 10% !
  • enhancement : LZNIB Normal & Optimal1 parsers much improved, more compression and faster decodes!
  • enhancement : LZNIB decodes faster
  • enhancement : LZNA about 10% faster to decode
  • enhancement : LZHLW decoder about 10% faster in 64-bit
  • enhancement : LZB encoder must faster in 32-bit
  • enhancement : LZNA on iOS now uses NEON, almost 2X faster to decode!
  • deprecation : LZH and LZA compressors are now deprecated; use LZHLW and LZNA instead
  • PS4 library now built with SDK 2.5
  • Windows library now built with MSVC 2013
  • Mac & iOS libraries now built with XCode 7.1

Release 2.1.0 - Oct 10, 2015

  • new : Oodle2 first Beta release

Release 1.45.1 - May 13, 2015

  • fix : fix bug in the LZNA level 7 (Optimal3) encoder

Release 1.45.0 - May 11, 2015

  • new : New LZNA compressor! More compression than LZA and fast to decode on modern CPUs.

Release 1.44.2 - March 4, 2015

  • enhancement : LZA new optimal parse; faster at Optimal1 and more compression at Optimal3
  • enhancement : LZNib Normal encoder faster, better optimal parse
  • enhancement : LZB Fast & VeryFast encoder faster

Release 1.44.1 - December 9, 2014

  • fix : example_packet state header was not the same in 32-bit and 64-bit builds
  • enhancement : example_packet LZ compression level option added

Release 1.44.0 - November 16, 2014

  • new : Mac sdk now includes 64 bit libraries
  • new : Android and IOS sdk's available, for Oodle Network only
  • new : Unreal Engine integration of Oodle Network is available
  • change : example_packet has better trained model IO

Release 1.43.0 - September 8, 2014

  • fix : LZA packet compressor introduced in 1.42 was getting much less compression than it should due a bug; fixed!

Release 1.42.0 - September 3, 2014

  • change : reduce memory use of LZNib and LZBLW compressors at Optimal level
  • new : LZA compressor now has sliding window encoding and decoding
  • new : OodleLZ_CompressWithContext incremental API now supports LZA
  • change : example_packet LZA option added
  • change : Oodle is using the new "radtypes.h" shared header
  • enhancement : OodleNetwork1 UDP new implementation - faster, a little more compression.
  • fix : Optimal parse compressors could hang on very large degenerate files

Release 1.41.0 - July 10, 2014

  • new : OodleNetwork dictionary selection by string matching; see OodleNetwork1_SelectDictionaryFromPackets and example_packet ; usually improves compression
  • enhancement : OodleNetwork1 TCP memory use reduced to 84104 bytes per channel
  • enhancement : OodleNetwork1 UDP is faster to encode ; saved states are incompatible with older versions
  • enhancement : LZNIB VeryFast and Normal much faster to encode; see Oodle_FAQ_LZCompareTable for speeds
  • enhancement : LZB16-Fast and VeryFast much faster to encode; see Oodle_FAQ_LZCompareTable for speeds
  • change : rename OodleNetwork1 TCP variant to OodleNetwork1TCP for clarity
  • change : better default hash table sizes for LZB16
  • fix : Fix bug in which packets that expanded in OodleNetwork1 TCP could cause a write at compbuf[-1]

Release 1.40.0 - June 27, 2014

  • change : OodleStaticLZP renamed to OodleNetwork1
  • enhancement : LZHLW compressor significantly faster to encode at all levels, especially Normal
  • new : New compressor LZA for very high compression!
  • fix : example_packet had a hard-coded limit of 64k for packet sizes; that was removed
  • fix : OodleLZDecoder_Reset was in the header but not implemented
  • new : OodleNetwork1 compressed buffer size is now limited by OodleNetwork1_CompressedBufferSizeNeeded

Release 1.30.0 - April 14, 2014

  • new : Minimal library initialization; see OodleX_Init_GetDefaults_Minimal , OodleX_Init_NoThreads , OodleX_Shutdown_NoThreads
  • change : OodleX_Init no longer accepts NULL options
  • change : Windows : OodleX_Init no longer does timeBeginPeriod(1) ; it's now left up to the app to decide if they want to do that
  • change : api simplified; many unused APIs removed

Release 1.22.0 - March 31, 2014

  • new : redo docs for Oodle Network packet compression

Release 1.21.2 - March 11, 2014

  • fix : OOP tool failed to extract large packages due to exceeding threading limits
  • fix : SPU LZ match offsets fixed again
  • fix : ThreadProfiler code stripped completely in final build

Release 1.21.0 - March 1, 2014

  • new : OodleNetwork1UDP_StateCompacted added to persist the trained state more compactly
  • enhancement : OodleLZH and LZHLW decompressors further optimized; 5-10% faster
  • enhancement : OodleLZH and LZHLW no longer does an allocation to prevent overrun
  • enhancement : OodleNetwork1 for UDP networking significantly faster (about 300%)
  • fix : SPU LZ match offsets could be too large by 1, causing corruption in rare cases

Release 1.20.0 - November 25, 2013

  • new : Oodle now shipping "final" build without debug facilities
  • new : OodleLZ_GetZipLikeCompressionSettings to make it easier to select compressor settings
  • fix : fix possible mismatch between structs in the public header and structs in the lib
  • fix : matchTableSizeLog2 wasn't being used correctly; made LZB and LZNIB compress less than they should

Release 1.19.0 - November 14, 2013

  • new : OodleNetwork1 now provides a UDP (stateless) variant for unordered packet compression (beta)
  • new : OodleHuffman_MultiHuffEncode for UDP unordered packet compression (beta)
  • note : version 119 numbering adjusted to match its intention better. 1.major.minor
  • fix : LZNib could fail when using "in-place" decompression
  • fix : LZH and LZHLW could speculatively read past the end of compbuf in rare cases, potentially causing a (benign) access violation
  • fix : LZ parallel compression with long-range-matcher could fail when parallel chunks didn't align to LRM chunks

Release 1.1.8 - November 4, 2013

  • new : add OodleXLZ_Decompress_MakeSeekTable_Wide_Async , a simple parallel version of OodleLZ_Decompress
  • new : add matchTableSizeLog2 to OodleLZ_CompressOptions ; controls encoder memory use and speed
  • new : add Oodle_FAQ_LZCompareTable to make choosing a compressor and level easier
  • enhancement : LZB16 encoder faster
  • fix : LZ decoders could speculatively read past the end of compressed buffers; this could cause an access violation if the memory after the compressed buffer was not readable. Fixed.
  • fix : Fix issues with the WithContext family of functions when used with circular window encoding.
  • fix : Fix issues with combining heterogenous OodleLZ data (with different seek chunk settings) causing the decoder to make invalid parallel decodes
  • fix : Fix issues with LZB when used with packed-raw-overlap decode mode
  • enhancement : PS4 : OodleMalloc goes to "default" memory instead of heap memory for large allocations

Release 1.1.7 - October 23, 2013

  • new : unified OodleLZ_CompressContext functions for incremental encoding, with and without sliding window
  • new : LZB16 can encode & decode in a circular window
  • fix : fix scheduling bug in OOZ IO->Decompression loader
  • fix : Linux : fix LZB16 generation page faults due to movdqa gcc bug
  • fix : don't lock Oodle threads to cores except on platforms where it is necessary

Release 1.1.6 - October 7, 2013

  • new : new LZB compressor - LZ-Bytewise (and LZB-LargeWindow), the fastest of all, for when even LZNib isn't fast enough
  • fix : OodleLZ_FillSeekTable could incorrectly mark heterogenous streams as being entirely independent seek chunks
  • fix : Linux & Mac : Fix worker thread stack sizes too small, could crash OOP tool and other uses of the worker threads.
  • fix : In-place decompression of incompressible data could fail if source overlapped dest.
  • fix : Fix OodleLZ_Compress_Async not working (and related APIs, such as OodleLZ_Compress_WriteOOZ_Async) when OodleAsyncSelect_None was passed for asyncSelect (now runs synchronously on the calling thread).

Release 1.1.5 - August 8, 2013

  • enhancement : LZNib now has a sliding window encoder for streaming compression
  • new : new OodleHuffman functions expose huffman-only coding
  • new : OodleNetwork1 compressor, static dictionary ideal for MMO network compression (Beta)

Release 1.1.4 - May 6, 2013

  • enhancement : large IO ops are now broken on the IO thread instead of at call time
  • enhancement : large IO aps are now cancellable in progress and return correct status
  • fix : OOP header could fail to read when it had a seek table
  • enhancement : rev the OOZ header; add the size of the header

Release 1.1.3 - May 1, 2013

  • fix : OOP header reader was truncating 64-bit file sizes to 32-bit
  • new : lzadvanced_output_chunking in example_lz_advanced
  • enhancement: OodleLZ_GetCompressedStepForRawStep allowed to seek quanta

Release 1.1.2 - April 28, 2013

  • fix : Linux dir enumeration was including "."
  • fix : Linux Open for WriteCreate was overwriting existing files too aggressively
  • fix : Linux Open failure had debug perror left in
  • change : better default log dirs and log names
  • change : log file will open in "." if default log dir is not accessible
  • change : Linux & Mac do log->prev cycle like Windows
  • change : disable stack trace on .map file platforms for now
  • enhancement: OodlePackage_FillWithFiles only decompresses chunks it needs (not all)
  • enhancement: OodlePackage_SortFiles now uses extention file groups

Release 1.1.1 - April 24, 2013

  • fix : Oodle_GetExtensionKey was not doing tolower as it should
  • fix : OodlePackage_FillWithFiles could crash if files failed to load
  • fix : fix memory overrun crash in DirListing
  • fix : fix null deref in arg parse in OOP tool
  • change: rename IOQ_DeleteFile_ to IOQ_Delete_ and make it work on dirs as well
  • new: add OodleDirListing_DeleteDirContents

Release 1.1.0 - April 11, 2013

  • new: LZNIB super fast to decode LZ variant added
  • new: LosslessFilters for some common data transformations that improve compression
  • change: Low level IO now allows aligned and unaligned IO modes
  • enhancement: Compression seek chunk size now variable, set by client
  • enhancement: All compressors can run encode and decode in parallel
  • enhancement: Long Range Matcher for LZ on big buffers
  • enhancement: (PS3) SPU waits now without thread switches
  • enhancement: (PS3) SPU after_decompress replaceable function

Release 1.0.0 - August 24, 2012

  • Initial Release.

 
   

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXLogCallbackRet
Navigation
 Welcome to Oodle
 Change Log
// Function typedef:
OodleXLogCallbackRetRet( OODLE_CALLBACK OodleXLogCallbackRet )( const char * buffer );
Discussion
Function pointer for OodleXLog_SetCallback
Parameters
buffer  the log message
Return Value
return  whether to supress the message or not
Discussion

OodleXLogCallbackRet is provided by the client to take log messages. It is called before other log outputs so that it has the chance to return OodleXLogCallbackRetRet_Terminate and supress other output.
 

OodleXLog_PrintfErrorOodleX Debug aidsOodleX Utils

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Shutdown_LogLeaks
Navigation
 OodleX_Init_GetDefaults_Threads
 OodleX_Shutdown_DebugBreakOnLeaks
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleX_Shutdown_LogLeaks
{
    OodleX_Shutdown_LogLeaks_No = 0,
    OodleX_Shutdown_LogLeaks_Yes = 1,
    OodleX_Shutdown_LogLeaks_Force32 = 0x40000000
};
Discussion
bool enum
 
OodleX_Init_GetDefaults_ThreadsOodleX Startup and ShutdownOodleX_Shutdown_DebugBreakOnLeaks

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
t_fp_OodleNet_Plugin_DisplayAssertion
Navigation
 t_fp_OodleNet_Plugin_Printf
 Welcome to Oodle
 Change Log
// Function typedef:
OODEFFUNC OO_BOOL( OODLE_CALLBACK t_fp_OodleNet_Plugin_DisplayAssertion )( const char * file,
                   const int line,
                   const char * function,
                   const char * message );
Discussion
Function pointer to Oodle Core assert callback
Parameters
file  C file that triggered the assert
line  C line that triggered the assert
function  C function that triggered the assert (may be NULL)
message  assert message
Return Value
return  true to break execution at the assertion site, false to continue
Discussion

This callback is called by Oodle Core when it detects an assertion condition.

This will only happen in debug builds.


 

t_fp_OodleNet_Plugin_PrintfNetwork plugins 

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXMalloc_OS_Options
Navigation
 Welcome to Oodle
 Change Log
// Enumerant:
enum OodleXMalloc_OS_Options
{
    OodleXMalloc_OS_Options_None = 0,
    OodleXMalloc_OS_Options_GuardBig = 1,
    OodleXMalloc_OS_Options_GuardBoth = 2,
    OodleXMalloc_OS_Options_GuardFrees = 3,
    OodleXMalloc_OS_Options_Count = 4,
    OodleXMalloc_OS_Options_Force32 = 0x40000000
};
Discussion
Options enum for OodleXMalloc_GetVTable_OS
Enumerants
OodleXMalloc_OS_Options_None  default
OodleXMalloc_OS_Options_GuardBig  guard page for big allocs
OodleXMalloc_OS_Options_GuardBoth  guard page for big and small allocs
OodleXMalloc_OS_Options_GuardFrees  GuardBoth + leak frees and make them NOACCESS
OodleXMalloc_OS_Options_Count 
OodleXMalloc_OS_Options_Force32 

 
About OodleXMallocOodleX Memory AllocatorsOodleXMallocVTable

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXHandleEvent_SetError
Navigation
 OodleXHandleEvent_SetDone
 OodleXHandleCountdown_Alloc
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleXHandleEvent_SetError( OodleXHandle h );
Discussion
Set an OodleXHandleEvent to OodleXStatus_Error
Parameters
h  handle created by OodleXHandleEvent_Alloc
Discussion

The state transition from Pending->Error is one way. If the handle is OodleXHandleAutoDelete_Yes, it goes away now.
 

OodleXHandleEvent_SetDoneOodleX async handle operationsOodleXHandleCountdown_Alloc

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
About Oodle on Xbox One
Navigation
 About Oodle on Mac
 About Oodle on Platforms
 About Oodle on Linux
 Welcome to Oodle
 Change Log

Oodle for XBox One is provided as a lib.

lib/oo2core_xboxone.lib
lib/oo2ext_xboxone.lib

The debug build of the Oodle lib is also provided. Generally the release build of Oodle should be linked with all versions of your game (do not link the debug build of Oodle with the debug build of your game typically). The debug build of Oodle is provided to help you track down problems.


since Oodle 2.8.5 :

Oodle for XBox One is built using MSVC 2017, and can be used with MSVC 2015-2019.

Oodle is provided for both XDK and GDK.

Oodle Core for XBox One does not use any stdio. The default log plugin outputs only to OutputDebugString.


Do not decompress directly into uncached graphics memory. See FAQ: How do I decompress to graphics memory quickly?.

Oodle now automatically detects this and warns about it.
 

About Oodle on MacAbout Oodle on PlatformsAbout Oodle on Linux

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OODLE_HEADER_VERSION
Navigation
 t_fp_Oodle_Job
 OodleNetworkVersion
 Welcome to Oodle
 Change Log
// Preprocessor definition:
#define OODLE_HEADER_VERSION ((46<<24)|(OODLE2_VERSION_MAJOR<<16)|(OODLE2_VERSION_MINOR<<8)|(OO_U32)sizeof(OodleLZ_SeekTable))
Discussion
OODLE_HEADER_VERSION is used to ensure the Oodle header matches the lib. Don't copy the value of this macro, it will change when the header is rev'ed.
Discussion
This is what you pass to OodleX_Init or Oodle_CheckVersion
 
t_fp_Oodle_JobCore BaseOodleNetworkVersion

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleX_Semaphore_Wait
Navigation
 OodleX_Semaphore_Post
 OodleX_CreateThread
 Welcome to Oodle
 Change Log
// Function prototype:
void OodleX_Semaphore_Wait( OodleX_Semaphore * sem );
Discussion
OodleX_Semaphore_Wait
Discussion
NOTE : it is not intended that you use these in production. They are for use in the Oodle examples. Replace with your own thread functions for shipping.
 
OodleX_Semaphore_PostOodleX threading utilOodleX_CreateThread

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_NameIsDir_AsyncAndWait
Navigation
 OodleXIOQ_GetFileSize_AsyncAndWait
 OodleX_GetOSFileOps
 Welcome to Oodle
 Change Log
// Function prototype:
OO_BOOL OodleXIOQ_NameIsDir_AsyncAndWait( const char * vfsName,
                                          OodleFileNotFoundIsAnError fnfiae );
Discussion
Convenience version of OodleXIOQ_GetInfoByName_AsyncAndWait
 
OodleXIOQ_GetFileSize_AsyncAndWaitOodleX low level async ioOodleX_GetOSFileOps

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleXIOQ_Fence_Async
Navigation
 OodleXIOQ_SetVTable
 OodleXIOQ_OpenForRead_Async
 Welcome to Oodle
 Change Log
// Function prototype:
OodleXHandle OodleXIOQ_Fence_Async( OodleXIOQFile fileRef OODEFAULT( 0 ),
                                    OodleXHandleAutoDelete autoDelete OODEFAULT( OodleXHandleAutoDelete_No ),
                                    OodleXPriority priority OODEFAULT( OodleXPriority_Normal ),
                                    const OodleXHandle * dependencies OODEFAULT( NULL ),
                                    OO_S32 numDependencies OODEFAULT( 0 ) );
Discussion
Add a "fence" to the operation queue
Parameters
fileRef  (optional) the file to associate the request with
autoDelete  (optional) lifetime of the operation handle ; see OodleXHandleAutoDelete
priority  (optional) priority of the operation ; see OodleXPriority
dependencies  (optional) dependencies; the async op won't start until these are all complete; note : these are not freed, they must be autodelete or you must free them some other way.
numDependencies  (optional) number of handles in deps array
Return Value
return  handle to the operation, or 0 if it could not be started (usually due to invalid args)
Discussion

A fence is a NOP which can be used to schedule against other operations. eg. if you have an OodleXIOQFile and want to block on any (unknown) operations on that file completing, you can add a Fence op to the file and block on it; earlier requests will flush first, so when the fence is done you know all previous requests are done.
 

OodleXIOQ_SetVTableOodleX low level async ioOodleXIOQ_OpenForRead_Async

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleNetwork1TCP_State
Navigation
 OodleNetwork1_Shared
 OodleNetwork1UDP_State
 Welcome to Oodle
 Change Log
// Structure definition:
struct OodleNetwork1TCP_State;
Discussion
Opaque data type for OodleNetwork1TCP_State
Discussion
This data is per-channel and adapts to the channel. There must be one for each encoder and one for each decoder.

This data is initialized either with OodleNetwork1TCP_State_InitAsCopy or OodleNetwork1TCP_State_Reset.

You can allocate and free it yourself. It must be of size OodleNetwork1TCP_State_Size.

For compression only of server->client data, your server must have one of these objects for each transmission channel (client). The client must have a matching one to receive from the server. They must be kept in sync - each one must get the same calls to Encode or Decode in the same order. If they ever get out of sync (eg. due to lost connection), then they must both be reset in the same way. (either OodleNetwork1TCP_State_InitAsCopy or OodleNetwork1TCP_State_Reset)


 

OodleNetwork1_SharedOodleAPI_OodleNetwork1OodleNetwork1UDP_State

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.
OodleLZ_GetInPlaceDecodeBufferSize
Navigation
 OodleLZ_GetDecodeBufferSize
 OodleLZ_GetCompressedStepForRawStep
 Welcome to Oodle
 Change Log
// Function prototype:
OO_SINTa OodleLZ_GetInPlaceDecodeBufferSize( OodleLZ_Compressor compressor,
                                             OO_SINTa compLen,
                                             OO_SINTa rawLen );
Discussion
Get the size of buffer needed for "in place" decode
Parameters
compressor  compressor used; OodleLZ_Compressor_Invalid to make it enough for any compressor
compLen  compressed data length
rawLen  decompressed data length
Return Value
return  size of buffer needed for "in place" decode ; slighly larger than rawLen
Discussion

To do an "in place" decode, allocate a buffer of this size (or larger). Read the compressed data into the end of the buffer, and decompress to the front of the buffer. The size returned here guarantees that the writes to the front of the buffer don't conflict with the reads from the end.

If compressor is one of the new codecs (Kraken,Mermaid,Selkie,Leviathan), the padding for in place decodes can be very small indeed. It is assumed you will be passing FuzzSafe_Yes to the decompress call.

If compLen is unknown, you want an in place buffer size that can accomodate any compressed data, then pass compLen = 0.

See OodleLZ_Decompress for more.
 

OodleLZ_GetDecodeBufferSizeOodleAPI_LZ_CompressorsOodleLZ_GetCompressedStepForRawStep

Oodle 2.9.11
E-mail oodle@radgametools.com for technical support.
Copyright © 2008-2023, Epic Games Tools LLC
All Rights Reserved.